Linux find files and total their size

1
find ./ -type f -newerct "1 May 2015" ! -newerct "1 Jul 2015" -print0 | du --files0-from=- -hc| tail -n1
Posted in Bash, Linux | Comments Off on Linux find files and total their size

Wireshark filters

Filter by ip dst or source using wildcard on the last 3 digits:

(ip.dst == 192.168.0.0/24) || (ip.src == 192.168.0.0/24)

Posted in Linux | Comments Off on Wireshark filters

MSBuild common errors and how to fix them

Building with MSBuild produced an error or warning

Run MSBuild with (d)etailed verbose mode and capture the output to a file:

1
msbuild someproject.csproj /t:Clean;Build;Transfer /p:OutputPath=bin\autobuild;BuildNumber=-1;Configuration=Debug /v:d >build.txt

Open build.txt and search for the MSBxxxx error or warning code. For example:

1
2
Consider app.config remapping of assembly "DotNetOpenAuth.AspNet, Culture=neutral, PublicKeyToken=2780ccd10d57b246" from Version "4.0.0.0" [] to Version "4.1.0.0" [C:\somedir\bin\DotNetOpenAuth.AspNet.dll] to solve conflict and get rid of warning.
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Microsoft.Common.targets(1605,5): warning MSB3247: Found conflicts between different versions of the same dependent assembly.

If it doesn’t already exist, add an App.config file with the following content:

1
2
3
4
5
6
7
8
9
10
11
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <runtime>
        <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
            <dependentAssembly>
                <assemblyIdentity name="DotNetOpenAuth.AspNet" publicKeyToken="2780ccd10d57b246" />
                <bindingRedirect oldVersion="0.0.0.0-4.1.0.0" newVersion="4.1.0.0" />
            </dependentAssembly>
        </assemblyBinding>
    </runtime>
</configuration>

Also change your .csproj to include the App.config file:

1
2
3
<ItemGroup>
    <None Include="App.config" />
</ItemGroup>

A few useful additions.

This fix applies to classic .NET Framework, not .NET Core / .NET 5+. Binding redirects in App.config are a .NET Framework concept. Modern .NET (Core 3.x, 5, 6, 7, 8, 9, …) resolves assembly versions through *.deps.json and unifies them automatically — you don’t write binding redirects, and MSB3247 generally doesn’t show up the same way. If your project is <Project Sdk=”Microsoft.NET.Sdk”> rather than the legacy XML format, you’re on modern .NET and this post doesn’t apply. 🧭

A better way to inspect a build than reading build.txt. Pass /bl (or -bl) to MSBuild instead of (or in addition to) /v:d:

1
msbuild someproject.csproj /t:Build /bl

This produces an msbuild.binlog file with the full structured build log. Open it in the free MSBuild Structured Log Viewer and you get a navigable tree of every target, task, and property, with full search — a vastly better experience than grepping through a text file. /v:diag is the next step up in plain-text verbosity if you don’t want a binlog (and is what the docs call “diagnostic” level).

Two other MSBuild errors that come up a lot.

MSB3644 — “Reference assemblies for .NETFramework,Version=vX.X were not found”. Classic on a fresh build agent. The targeting pack for the .NET Framework version your project asks for isn’t installed. Two fixes: install the matching .NET Framework Targeting Pack via the Visual Studio Installer (Individual components → “.NET Framework X.X targeting pack”), or downgrade <TargetFrameworkVersion> in your .csproj to a version that is installed.

MSB4018 — “The X task failed unexpectedly”. The MSBuild equivalent of “something blew up, here’s the stack trace”. Usually one of: a custom task that crashed, an SDK version mismatch (your global.json pins an SDK that isn’t on the machine), or a corrupted obj/ folder from an interrupted build. The standard checklist: delete bin/ and obj/, run dotnet restore (or nuget restore for legacy), confirm dotnet –version matches what global.json demands, then rebuild. If it still fails, the binlog from the previous tip will show you which task threw — that’s nearly always the real clue. 🛠️

Posted in C# | Tagged | Comments Off on MSBuild common errors and how to fix them

Allowing a linux/unix user all sudo access without password

/etc/sudoers

yourusername ALL=(ALL) NOPASSWD:ALL

Posted in Linux | Tagged , | Comments Off on Allowing a linux/unix user all sudo access without password

Solaris Notes

Extracting a solaris package without installing it.

#http://serverfault.com/questions/287469/extract-files-out-of-solaris-pkg-file-without-installing
pkgtrans filename.pkg /home/user/temporary_package_prefix

Posted in Operating System | Comments Off on Solaris Notes

Find with xargs

Search all files under current directory, look for xml node ‘‘, copy and print that node. Send the output to a file.

find . |xargs -n1 xmlstarlet sel -t -c “//processorInfo[@ruleType=’store’]” 2>/dev/null > /cygdrive/b/allstore.txt

Posted in Linux | Comments Off on Find with xargs

XMLStartlet – Command line xml queries

xmlstarlet sel -t -c “//YOUR_NODE_ELEMENT_TAG_NAME[@ATTRIBUTE_NAME=’ATTRIBUTE_VALUE’]” YOUR_XML_FILE.xml

Posted in Linux | Comments Off on XMLStartlet – Command line xml queries

Bash Scripting templates

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
# ==========================
# Bash tips
# ==========================

# Run bash by
$ /bin/bash scriptname.sh
$ /bin/bash scriptname.sh +x  # debug mode

# Or put the next line on the 1st line of your file.
#!/bin/bash -x

## Double round braket for integer
if (( $MYINT < 10 ]]   #or: =
## Double square braket for string
if [[ $USER = "ron" ]]   #or: !=
   then
      echo "Hello ron"
   else
      echo "Hello Stranger"
fi

$ date +%x   # shows only the date 07/02/13 depending on locale, currently en_CA
$ LANG=en_US
$ date +%x   # shows only the date 02/07/13


# ==========================
# vi tips
# ==========================

# Using normal mode
# Delete line at the cursor
#    type dd
# Insert at cursor
#    type i
# Insert after the cursor (append)
#    type a
k
J
o

# To bring in (read in) the content of an extrnal file into your current file
:r ~/snippets/if

# To read in a result of an executable, such as date use the :r!
:r!date
:r!date "+\%x"

# Macro normal mode
:map
# Macro insert mode
:map!

# Macro F2 to insert text: "#This file was created on 2014-05-03"
:map <F2> i#This file was created on <ESC>:r!date "+\%x" <ESC> kJ

put the line above to .vimrc
map <F2> i#!/bin/bash<ESC>
map <F3> o#This file was created on <ESC>:r!date "+\%x" <ESC> kJ
Posted in Linux | Comments Off on Bash Scripting templates

Python Sample Script

You inherit a folder full of bash scripts. They source each other, they call each other through pipelines, and after a while nobody remembers who depends on whom. You want a picture — “if I touch deploy.sh, what else might break?” — without reading every file by hand. This little Python script does exactly that: it scans a directory of .sh files, finds every reference one script makes to another, and renders the relationships as a GraphViz diagram. 🐍

What you’ll need

  • Python 3 — the code below runs as-is on a modern Python 3 (it was originally written for Python 2 but uses no incompatible idioms, so just python3 yourscript.py)
  • Graphviz — provides the dot command-line tool that turns DOT notation into an image. Install with sudo apt install graphviz (Debian/Ubuntu), sudo dnf install graphviz (Fedora/RHEL), or brew install graphviz (macOS).
  • An SVG viewer — the script ends by opening the result in eog (Eye of GNOME). On macOS, swap that for open; on Windows, start.

No third-party Python libraries are needed — everything (re, glob, ntpath, subprocess) is in the standard library.

Step 1: scan a single bash file for dependencies

The first job is to open one bash file and find every .sh filename mentioned inside it. We use a regex to grab anything that looks like a path ending in .sh, then walk backwards from each match to the start of the line to check whether the match is inside a comment or a quoted string — if it is, we skip it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
import re
import ntpath
import glob
import subprocess

# In order to get this tool to run correctly, you'll need python installed (duh!) and graphviz.
basedir = "/home/ronald-dev/allBashScript/"

'''
This function will scan a bash file, and look for any dependency to another bash file
@param String    The fileName
@return Array    The dependency filenames
'''

def readDependencies(fileName):
    f = open(fileName, 'r')
    content = f.read()
    occurences = {}  # {file1.sh:1, file2.sh:1, file3.sh:1}
    for m in re.finditer('[\w|\/]*\.sh', content):
        word = ntpath.basename(content[m.start(): m.end()])
        if word not in occurences:
            isComment = False
            index = m.start()
            while index > 0 and content[index] != "\n":
                if content[index] == """ or content[index] == "'" or content[index] == "#":
                    isComment = True
                    break
                else:
                    index = index - 1
            if isComment == False:
                occurences[word] = 1
    f.close()
    return occurences.keys()

The dictionary trick (occurences[word] = 1) is just an old-school way of getting a unique set of names — every file only counts once, no matter how many times it’s referenced. Today you’d reach for a set() instead, but the result is the same. Change basedir to wherever your bash scripts live.

Step 2: turn dependencies into GraphViz DOT notation

GraphViz speaks a tiny language called DOT. A directed edge from a.sh to b.sh looks like “a.sh” -> “b.sh”;. Wrap a list of those in digraph G { … } and you have a complete graph. We need a function that turns one file’s dependencies into those edges, and a driver that does it for the whole directory.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
'''
Scan one bash file and convert its dependencies to GraphViz notation
@param String    The fileName
@param Array     The dependency filenames
@return String   GraphViz notation "bash01.sh" -> "bash02.sh";[NEWLINE] etc
'''

def toGraphvizNotation(fileName, arrayOfDependencies):
    string = ""
    filename = ntpath.basename(fileName)
    for dependency in arrayOfDependencies:
        string += '    "' + filename + '" -> "' + dependency + "";\n"
    return string

'''
Scans all bash files, with extension .sh
@param String    The directory to scan
@return Array    Array of filenames
'''
def readAllBashFiles(directory):
    return glob.glob(directory + "
*.sh")

'''
Scan multiple bash files and convert their dependencies to GraphViz notation
@param String    The directory to scan
@return String   GraphViz notation "
bash01.sh" -> "bash02.sh";[NEWLINE] etc
'''
def generateGraphVizFile(basedir):
    bashfiles = readAllBashFiles(basedir)
    graphVizFileContent = "
"
    for bashfile in bashfiles:
        graphVizFileContent += toGraphvizNotation(bashfile, readDependencies(bashfile))
    graphVizFileContent = "
digraph G { \n" + graphVizFileContent + "}"
    return graphVizFileContent

At this point we can turn the whole directory into a single block of DOT text — no files written yet, no images rendered, just a string that looks like:

1
2
3
4
5
digraph G {
    "deploy.sh" -> "common.sh";
    "deploy.sh" -> "db_backup.sh";
    "db_backup.sh" -> "common.sh";
}

Step 3: write the .dot file, render it, and open the picture

The last step has three parts: write the DOT string to a file, shell out to dot to convert it to SVG, then open the SVG in an image viewer. Three lines, one for each.

1
2
3
4
5
6
7
8
9
10
11
12
13
'''
Write content to a file
@param String    The filename to be written to
@param String    The content
'''

def writeGraphvizFile(filename, graphVizFileContent):
    f = open(filename, 'w')
    f.write(graphVizFileContent)
    f.close()

writeGraphvizFile("/tmp/bashdependencies.dot", generateGraphVizFile(basedir))
subprocess.call(["/usr/bin/dot", "-T", "svg", "-o", "/tmp/allBashScript.svg", "/tmp/bashdependencies.dot"])
subprocess.call(["eog", "/tmp/allBashScript.svg"])

Run the script and you should see your dependency graph open in an image viewer. If you’d rather just produce the SVG without opening it, drop the last subprocess.call line.


A few things worth knowing.

Python 3 vs Python 2. The original was written in 2014, when Python 2 was still everyone’s default. The functional code above runs unchanged on Python 3 — there are no print statements, no xrange, no implicit str/bytes mixing. Add a shebang and you’re done:

1
#!/usr/bin/env python3

The dot path is hard-coded. /usr/bin/dot works on most Linux distros but not on macOS (where Homebrew puts it under /opt/homebrew/bin/ on Apple Silicon, or /usr/local/bin/ on Intel) and not on Windows. Drop the absolute path and let $PATH resolve it:

1
subprocess.call(["dot", "-T", "svg", "-o", "/tmp/allBashScript.svg", "/tmp/bashdependencies.dot"])

Same idea for the viewer: eog is GNOME-only. Cross-platform, the standard trick is:

1
2
3
import sys
opener = {"darwin": "open", "win32": "start"}.get(sys.platform, "xdg-open")
subprocess.call([opener, "/tmp/allBashScript.svg"])

The regex has a small bug. The pattern [\w|\/]*\.sh uses a character class that includes | as a literal pipe — not as alternation, because | has no special meaning inside square brackets. It still works in practice (pipes rarely appear next to .sh), but the cleaner intent is [\w/]*\.sh.

The comment-detection misses quoted dependencies. The script walks backwards looking for #, , or to decide whether a match is inside a comment or string — and skips it if it is. That’s correct for # source helper.sh, but it also skips legitimate dependencies like source “common.sh” or bash ‘helper.sh’, because the trailing quote on the same line trips the check. If your scripts wrap filenames in quotes, you’ll get a graph that’s missing edges. A more robust approach is to strip comments first (everything after a non-quoted #), then run the regex on the cleaned content.

Modernise the file I/O. The pattern f = open(…); … ; f.close() works but won’t close the file if anything in between throws. Use with:

1
2
with open(fileName, 'r') as f:
    content = f.read()

And of course, in 2026 you might just ask a smart code-search tool for the same picture — but there’s still something satisfying about a script you can read end to end in five minutes. 🌳

Posted in Python | Comments Off on Python Sample Script

Enabling SSH (https) for Apache 2 in Ubuntu/Mint/Possibly other Debian distro

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
# Intall apache
$ sudo apt-get install apache2
 
# Enable SSL module
$ sudo a2enmod ssl

# Restart Apache
$ sudo service apache2 restart

# Create a directory to store the SSLCertificateFile and SSLCertificateKeyFile
$ mkdir  /etc/apache2/ssl

# Generate the keys
$ sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/apache2/ssl/apache.key -out /etc/apache2/ssl/apache.crt
Generating a 2048 bit RSA private key
...................................+++
................................+++
writing new private key to '/etc/apache2/ssl/apache.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:CA
State or Province Name (full name) [Some-State]:Ontario
Locality Name (eg, city) []:Ottawa
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Ronald Pringadi      
Organizational Unit Name (eg, section) []:Engineering
Common Name (e.g. server FQDN or YOUR name) []:ronald-mint.com
Email Address []:webmaster@some-cool-website.com

# Edit your hosts file and add "127.0.0.1    some-cool-website.com"
$ gedit /etc/hosts


# Edit your sites-available ssl config
$ gedit /etc/apache2/sites-available/default-ssl.conf
#    Make sure you add the following line under the email
     ServerName some-cool-website:443
#    Also replace
     SSLCertificateFile /etc/apache2/ssl/apache.crt
     SSLCertificateKeyFile /etc/apache2/ssl/apache.key

# Activte site
$ sudo a2ensite default-ssl

# Reload Apache
$ sudo service apache2 reload

Open your browser and point it to https://some-cool-website.com. The browser will prompt you that the website is using a self-sign certificate, and do you want to continue and accept that certificate. Answer yes and that’s all to it

Posted in Linux, Ubuntu, Web Development | Comments Off on Enabling SSH (https) for Apache 2 in Ubuntu/Mint/Possibly other Debian distro