Introduction to Logging and Tracing in PHP

If you still tracing variable in php using print_r() or var_dump() then you’ll see them directly in the screen or the web page that you’re working on. This is easy to see, but also prone to problems:

  1. What if visitors see your error trace? Surely you don’t want that.
  2. What if you have a function call that relies on no string has been printed yet, e.g.: header(), ini_set(), these functions will fail and depending on your settings most likely when they fail they don’t show any sign of error.

Best way to do code tracing in PHP is to write the following code into your file – one of the early-executed file, such as config/header file, where you don’t have any output yet:

1
2
3
ini_set('log_errors','On');
ini_set('error_log', '/var/wwwtmp/php.log');
error_reporting(E_ALL ^ (E_NOTICE | E_WARNING));

And at the part where you want to trace a certain variable

1
error_log(__FILE__.' ('.__LINE__.') '.__CLASS__.' :: '. __FUNCTION__ .' :: '. print_r($YOUR_VARIABLE_HERE,true));

By default apache2 and php will output their error in /var/log/apache2/error.log ,but after executing the ini_set above, the error log will be produced in /var/wwwtmp/php.log. And you need to make sure you that directory exists, if not use other directory that www-data has write access to. Why do we need to setup in a different location? Because the default file output doesn’t support showing new lines, you’ll see lot’s of \n. Thus make tracing hard to read.

Posted in PHP, Web Development | Comments Off on Introduction to Logging and Tracing in PHP

Robocopy – Automate Your Windows Copy

I backup my files regularly using the Windows scheduler. For example, several folders from my Windows preferences folder on drive C (C:\Users\dellxps\AppData\Roaming) to another drive, D:\Users\Ronald\AppData\Roaming.

There is a built-in smart copier in Windows that not many people know about, called robocopy. Take a look at the following scripts: backup.bat and restore.bat.

backup.bat

1
2
3
4
5
6
7
8
@echo off
SET vSourceRootFolder=C:\Users\dellxps\AppData\Roaming
SET vDestinationRootFolder=D:\Users\Ronald\AppData\Roaming
echo "========================================"
echo "== Backup application preference files"
echo "========================================"
robocopy %vSourceRootFolder%\"FileZilla\" %vDestinationRootFolder%\"FileZilla\" /MIR /R:2 /W:0 /XJD /XJ /NP /LOG+:D:\sync\backup_d_drive.log
robocopy %vSourceRootFolder%\"Mozilla\" %vDestinationRootFolder%\"Mozilla\" /MIR /R:2 /W:0 /XJD /XJ /NP /LOG+:D:\sync\backup_d_drive.log

restore.bat

1
2
3
4
5
6
7
8
@echo off
SET vDestinationRootFolder=C:\Users\dellxps\AppData\Roaming
SET vSourceRootFolder=D:\Users\Ronald\AppData\Roaming
echo "========================================"
echo "== Restore application preference files"
echo "========================================"
robocopy %vSourceRootFolder%\"FileZilla\" %vDestinationRootFolder%\"FileZilla\" /MIR /R:2 /W:0 /XJD /XJ /NP /LOG+:D:\sync\backup_d_drive.log
robocopy %vSourceRootFolder%\"Mozilla\" %vDestinationRootFolder%\"Mozilla\" /MIR /R:2 /W:0 /XJD /XJ /NP /LOG+:D:\sync\backup_d_drive.log

A few useful additions.

The flag list in the script above is dense — here’s what each piece actually buys you in this context:

  • /MIR — “mirror” the destination to match the source. This deletes files at the destination that no longer exist in the source. Great for a backup where you want a faithful copy; dangerous if you accidentally swap source and destination. Always double-check the order of arguments.
  • /R:2 — retry up to 2 times on failure. The default is 1 million retries, which can hang a script for hours if a single file is locked. Lowering this is almost always the right move.
  • /W:0 — wait 0 seconds between retries. Default is 30 seconds. Combined with /R:2, you fail fast on a stubborn file instead of stalling.
  • /XJD and /XJ — exclude junction points (directory symlinks). Without these, robocopy can follow a junction and end up copying the entire system drive into your backup, which is a fun way to fill a disk.
  • /NP — no progress percentage. Keeps the log file clean (otherwise every percent update gets a line).
  • /LOG+:path — append to a log file. The + is what makes it append rather than overwrite — drop the + for a fresh log on every run.

A couple of safety habits worth adopting:

Test with /L first. The /L flag means “list only — don’t actually copy or delete anything.” Add it to your robocopy line, run the script, and inspect the log to see what would happen. Especially valuable when /MIR is involved, because the deletes it performs are silent and instant.

1
robocopy %vSourceRootFolder%\"FileZilla\" %vDestinationRootFolder%\"FileZilla\" /MIR /R:2 /W:0 /XJD /XJ /NP /L

Robocopy exit codes are weird. Anything 0–7 is success (0 = no files copied, 1 = files copied successfully, 2 = extra files at dest, etc.). Anything >= 8 is a real failure. If you wrap robocopy in a script that checks %ERRORLEVEL%, the natural if errorlevel 1 check will think every successful run is a failure. Use if %ERRORLEVEL% GEQ 8 instead.

1
2
3
4
5
robocopy %src% %dst% /MIR /R:2 /W:0
if %ERRORLEVEL% GEQ 8 (
    echo Robocopy failed.
    exit /b 1
)

Mirror in one direction, period. A common mistake is to schedule a backup script and a separate restore script that both run on a timer, thinking it’s bidirectional sync. It isn’t — whichever one ran most recently wins, and the other side gets overwritten. If you want bidirectional sync, you want a different tool (rsync over SSH, SyncThing, OneDrive, etc.) — robocopy is one-way only.

Below is the full robocopy /? reference for completeness. The flag set hasn’t meaningfully changed across Windows versions since Vista, so this is still accurate on Windows 10 and 11.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
-------------------------------------------------------------------------------
   ROBOCOPY     ::     Robust File Copy for Windows                              
-------------------------------------------------------------------------------

  Started : Wed Sep 19 21:48:29 2012

              Usage :: ROBOCOPY source destination [file [file]...] [options]

             source :: Source Directory (drive:\path or \\server\share\path).
        destination :: Destination Dir  (drive:\path or \\server\share\path).
               file :: File(s) to copy  (names/wildcards: default is "*.*").

::

:: Copy options :
::
                 /S :: copy Subdirectories, but not empty ones.
                 /E :: copy subdirectories, including Empty ones.
             /LEV:n :: only copy the top n LEVels of the source directory tree.

                 /Z :: copy files in restartable mode.
                 /B :: copy files in Backup mode.
                /ZB :: use restartable mode; if access denied use Backup mode.
            /EFSRAW :: copy all encrypted files in EFS RAW mode.

  /COPY:copyflag[s] :: what to COPY for files (default is /COPY:DAT).
                       (copyflags : D=Data, A=Attributes, T=Timestamps).
                       (S=Security=NTFS ACLs, O=Owner info, U=aUditing info).

           /DCOPY:T :: COPY Directory Timestamps.

               /SEC :: copy files with SECurity (equivalent to /COPY:DATS).
           /COPYALL :: COPY ALL file info (equivalent to /COPY:DATSOU).
            /NOCOPY :: COPY NO file info (useful with /PURGE).

            /SECFIX :: FIX file SECurity on all files, even skipped files.
            /TIMFIX :: FIX file TIMes on all files, even skipped files.

             /PURGE :: delete dest files/dirs that no longer exist in source.
               /MIR :: MIRror a directory tree (equivalent to /E plus /PURGE).

               /MOV :: MOVe files (delete from source after copying).
              /MOVE :: MOVE files AND dirs (delete from source after copying).

     /A+:[RASHCNET] :: add the given Attributes to copied files.
     /A-:[RASHCNET] :: remove the given Attributes from copied files.

            /CREATE :: CREATE directory tree and zero-length files only.
               /FAT :: create destination files using 8.3 FAT file names only.
               /256 :: turn off very long path (> 256 characters) support.

             /MON:n :: MONitor source; run again when more than n changes seen.
             /MOT:m :: MOnitor source; run again in m minutes Time, if changed.

      /RH:hhmm-hhmm :: Run Hours - times when new copies may be started.
                /PF :: check run hours on a Per File (not per pass) basis.

             /IPG:n :: Inter-Packet Gap (ms), to free bandwidth on slow lines.

                /SL :: copy symbolic links versus the target.

            /MT[:n] :: Do multi-threaded copies with n threads (default 8).
                       n must be at least 1 and not greater than 128.
                       This option is incompatible with the /IPG and /EFSRAW options.
                       Redirect output using /LOG option for better performance.

::

:: File Selection Options :
::
                 /A :: copy only files with the Archive attribute set.
                 /M :: copy only files with the Archive attribute and reset it.
    /IA:[RASHCNETO] :: Include only files with any of the given Attributes set.
    /XA:[RASHCNETO] :: eXclude files with any of the given Attributes set.

 /XF file [file]... :: eXclude Files matching given names/paths/wildcards.
 /XD dirs [dirs]... :: eXclude Directories matching given names/paths.

                /XC :: eXclude Changed files.
                /XN :: eXclude Newer files.
                /XO :: eXclude Older files.
                /XX :: eXclude eXtra files and directories.
                /XL :: eXclude Lonely files and directories.
                /IS :: Include Same files.
                /IT :: Include Tweaked files.

             /MAX:n :: MAXimum file size - exclude files bigger than n bytes.
             /MIN:n :: MINimum file size - exclude files smaller than n bytes.

          /MAXAGE:n :: MAXimum file AGE - exclude files older than n days/date.
          /MINAGE:n :: MINimum file AGE - exclude files newer than n days/date.
          /MAXLAD:n :: MAXimum Last Access Date - exclude files unused since n.
          /MINLAD:n :: MINimum Last Access Date - exclude files used since n.
                       (If n < 1900 then n = n days, else n = YYYYMMDD date).

                /XJ :: eXclude Junction points. (normally included by default).

               /FFT :: assume FAT File Times (2-second granularity).
               /DST :: compensate for one-hour DST time differences.

               /XJD :: eXclude Junction points for Directories.
               /XJF :: eXclude Junction points for Files.

::

:: Retry Options :
::
               /R:n :: number of Retries on failed copies: default 1 million.
               /W:n :: Wait time between retries: default is 30 seconds.

               /REG :: Save /R:n and /W:n in the Registry as default settings.

               /TBD :: wait for sharenames To Be Defined (retry error 67).

::

:: Logging Options :
::
                 /L :: List only - don't copy, timestamp or delete any files.
                 /X :: report all eXtra files, not just those selected.
                 /V :: produce Verbose output, showing skipped files.
                /TS :: include source file Time Stamps in the output.
                /FP :: include Full Pathname of files in the output.
             /BYTES :: Print sizes as bytes.

                /NS :: No Size - don't log file sizes.
                /NC :: No Class - don't log file classes.
               /NFL :: No File List - don't log file names.
               /NDL :: No Directory List - don't log directory names.

                /NP :: No Progress - don't display percentage copied.
               /ETA :: show Estimated Time of Arrival of copied files.

          /LOG:file :: output status to LOG file (overwrite existing log).
         /LOG+:file :: output status to LOG file (append to existing log).

       /UNILOG:file :: output status to LOG file as UNICODE (overwrite existing log).
      /UNILOG+:file :: output status to LOG file as UNICODE (append to existing log).

               /TEE :: output to console window, as well as the log file.

               /NJH :: No Job Header.
               /NJS :: No Job Summary.

           /UNICODE :: output status as UNICODE.

::

:: Job Options :
::
       /JOB:jobname :: take parameters from the named JOB file.
      /SAVE:jobname :: SAVE parameters to the named job file
              /QUIT :: QUIT after processing command line (to view parameters).
              /NOSD :: NO Source Directory is specified.
              /NODD :: NO Destination Directory is specified.
                /IF :: Include the following Files.
Posted in DOS, Operating System | Comments Off on Robocopy – Automate Your Windows Copy

Displaying Linux Distro Version and Showing 32Bit or 64Bit OS

Checking whether you have 32bit linux or 64bit linux. From the code below we can see that it is a 64 bit linux.

1
2
[user1@ubuntu 17:14:37 ~/dev]$ uname -mrs
Linux 3.2.0-30-generic x86_64

Checking which distro you’re using:

1
2
3
4
5
6
[user1@ubuntu 17:14:48 ~/dev]$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 12.04.1 LTS
Release:    12.04
Codename:   precise
Posted in Linux, Operating System | Comments Off on Displaying Linux Distro Version and Showing 32Bit or 64Bit OS

Find and Replace File Content using Linux Command Line

1
find . -name "*.pgsql" -print | xargs sed -i 's/STRINGTOREPLACE/REPLACERSTRING/g'
Posted in Linux, Operating System | Comments Off on Find and Replace File Content using Linux Command Line

How To List All Users or Groups in Linux

To list all users in Linux:

1
cat /etc/passwd

See only the first word before double colon (:)
for exampple, user “root” will look like: root:x:0:0:root:/root:/bin/bash

And to list all groups do the following

1
cat /etc/group
Posted in Linux, Operating System | Comments Off on How To List All Users or Groups in Linux

How Big is The Big Apple?

Do you know how much bigger is Apple compared to the other big companies in the US?
http://www.greenandredmarket.com/menu/biggestcompanies.htm
And scroll to the lower part of the page.

I’m not an apple fan boy but I can see that our society buys a lot of apple products: iPhones, iPad, iMac, MacBook Pro, Macbook Air.

Posted in Investment, Stock Market | Comments Off on How Big is The Big Apple?

Internet Explorer (IE) friendly table DOM manipulation

Occasionaly we can manipulate HTML DOM easily by accessing the innerHTML of an element. In IE however, the following elements’ innerHTML are read only: COL, COLGROUP, FRAMESET, HEAD, HTML, STYLE, TABLE, TBODY, TFOOT, THEAD, TITLE, TR.

This code will fail in IE

1
tBodyEl.innerHTML = '<tr height="24px"><td colspan="7">' + 'Some Cell Value' + '</td></tr>';

A more IE friendly solution using DOM table functions.

1
2
3
4
5
var row = tBodyEl.insertRow(0);
row.setAttribute('height', '24px');
var cell = row.insertCell(0);
cell.setAttribute('colspan', '7');
cell.innerHTML = 'Some Cell Value';
Posted in javascript, Web Development | Comments Off on Internet Explorer (IE) friendly table DOM manipulation

Preventing Another Instance of A Program Running in Linux

The code below is my solution for preventing a program from running more than once at the same time in Linux.
The scenario: Consider if you have a cron job (jobA.sh) set to run at 6PM, 7PM, and 8PM.
While the 1st instance of jobA.sh started, it might (or might not) need more than an hour to finish. If it does requires more than one hour, while the 1st instance is still running, a 2nd instance of jobA.sh started.
Now you ended up with two instances of a same program consuming your CPU, files, etc – which will slow things down even more.

So here how I solve it:
When the 1st instance is started, check if there is a temporary lock file, if yes then just exits or quit. If not create the temporary lock file. And do your code/algorithm. After it finishes delete this lock file. When the 2nd instance started it will also do the same thing, checking if there is lock file.
It will be wise to put another cron job that cleans all lock file every 00:00AM. Just to be sure the next execution (at 6PM tomorrow) will be able to run

Since I’m no bash expert, let me know if there is a better way to do this. Cheers!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
#!/bin/bash
# =================================
# author: Ronald Pringadi
# sleep command is in seconds
# =================================
vLockFile=/tmp/lockfile1.lock
echo 'start'
# If lockfile does not exist do the followings
if [ ! -f $vLockFile ]; then
    touch $vLockFile
    echo "Lock file doesn't exists, executing code"
    sleep 7 # this can be your big process that takes longer than an hour, but just for proof of concept, I use 7 seconds
    rm $vLockFile
else
    echo "Lock file exists do nothing"
fi
echo 'done'
Posted in Linux, Operating System | Comments Off on Preventing Another Instance of A Program Running in Linux

Extjs – Ext.apply

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
console.clear();
var a = {
    abc : 1,
    def : 2
};
var b = {
    abc : 1.1,
    def : 2.2
};

var c = Ext.apply( a , b ); //Ext.apply overwrites 1st variable (by ref) with the 2nd var

console.log('c');
console.log(c);

console.log('a');
console.log(a);

console.log('b');
console.log(b);

Result:
c
Object { abc=1.1, def=2.2}
a
Object { abc=1.1, def=2.2}
b
Object { abc=1.1, def=2.2}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
var a = {
    abc : 1,
    def : 2
};
var b = {
    abc : 1.1,
    def : 2.2,
    gij : 3.3
};

var d = Ext.applyIf( a , b ); //Ext.applyIf overwrites 1st variable (by ref) with the 2nd var, only if it doesn't exists
console.log('d');
console.log(d);

console.log('a');
console.log(a);

console.log('b');
console.log(b);

Result:
d
Object { abc=1, def=2, gij=3.3}
a
Object { abc=1, def=2, gij=3.3}
b
Object { abc=1.1, def=2.2, gij=3.3}

Posted in Ext Js, javascript, Web Development | Comments Off on Extjs – Ext.apply

MySQL Backup and Restore – Using Command Line

Here’s how you back up a database from the command line. The first line is the backup, the second is the restore.

These commands work on both Windows and Linux. On Windows you might want to add the MySQL bin directory to your environment PATH — the easy way is Win + Break → Advanced System Settings → Environment → PATH. Then open cmd (the command prompt). If you already had a command prompt open before adding the PATH, restart it for the change to take effect.

MySQL doesn’t export stored procedures, functions, or triggers by default. If you need those, add –routines –triggers to your command. Even if you don’t currently use stored procedures, functions, or triggers, it’s safer to back up with those options enabled — they’re cheap, and you’ll thank yourself later if someone adds one.

1
2
mysqldump -uusername -ppassword --routines --triggers --lock-tables=false database_name > YOURSQLFILE.sql
mysql -uusername -ppassword database_name < YOURSQLFILE.sql

If you need to back up all databases, the user we typically use is root. This will back up all triggers and functions. You’ll need to pre-create any other users with the appropriate privileges on stored procedures or functions for the restored copy to work end-to-end.

1
2
mysqldump -uusername -ppassword --opt --events --all-databases > alldb.sql
mysql -uusername -ppassword < alldb.sql

A few useful additions.

The password-on-the-command-line problem. The -ppassword form (with no space — that’s important) is convenient but has two real issues: anyone running ps aux on the same machine can see your password while the command is running, and it gets stored in your shell history. Two cleaner options:

1
2
3
4
5
6
7
8
9
10
11
12
# Option 1: prompt for the password interactively
mysqldump -uusername -p database_name > out.sql
# (you'll be asked: Enter password:)

# Option 2: use an option file with restricted permissions
cat > ~/.my.cnf << 'EOF'
[client]
user=username
password=secret
EOF

chmod 600 ~/.my.cnf
mysqldump --defaults-file=~/.my.cnf database_name > out.sql

The ~/.my.cnf approach also helps for scheduled backups, where there’s no human around to type the password. Just make sure the file mode is 600 — MySQL clients refuse to load it otherwise on Linux.

For InnoDB tables: –single-transaction. If your tables are InnoDB (the default since MySQL 5.5), this flag wraps the dump in a transaction and gives you a consistent snapshot without locking the tables. Concurrent writes can keep happening while you back up:

1
2
mysqldump --single-transaction --routines --triggers \
    -uusername -p database_name > out.sql

Don’t combine it with –lock-tables — they fight each other. –single-transaction only works for InnoDB; if you have MyISAM tables, use –lock-tables (the default for that case) and accept that writes block.

The MySQL 8 “Access denied PROCESS privilege” trap. Starting around MySQL 5.7+/8.0, mysqldump tries to query information_schema.tablespaces and fails for non-superusers with a confusing PROCESS-privilege error:

1
mysqldump: Error: 'Access denied; you need (at least one of) the PROCESS privilege(s) for this operation' when trying to dump tablespaces

The fix is one flag — –no-tablespaces. Most application backups don’t actually need tablespace info anyway:

1
2
mysqldump --no-tablespaces --single-transaction --routines --triggers \
    -uusername -p database_name > out.sql

Compress as you go. SQL dumps compress beautifully — typical 5-10x. For anything larger than a few hundred megabytes, pipe through gzip:

1
2
3
4
5
6
7
# backup
mysqldump --single-transaction --routines --triggers -uusername -p database_name | gzip > out.sql.gz

# restore
gunzip < out.sql.gz | mysql -uusername -p database_name
# or in one shot
zcat out.sql.gz | mysql -uusername -p database_name

Restoring a single database from an –all-databases dump. Common scenario: you have a full-instance dump but only want to restore one schema. mysql has a –one-database flag that ignores statements not for the named database:

1
mysql -uusername -p --one-database wantedschema < alldb.sql

Caveat: –one-database filters by the most recent USE statement, not by table name, so it works cleanly when the dump uses USE schema; blocks (which mysqldump –all-databases emits by default). For surgical restores from arbitrary dumps, sed/awk through the file is sometimes cleaner.

For really big databases: mysql shell. If you’re operating on databases in the hundreds-of-GB range, the modern mysqlsh utility (ships with MySQL 8.0+) does parallel chunk-based dumps and is dramatically faster than mysqldump:

1
2
mysqlsh -uusername -h hostname -- util dumpInstance /path/to/backup/dir --threads=4
mysqlsh -uusername -h hostname -- util loadDump /path/to/backup/dir --threads=4

It also handles compression natively, supports resumable loads, and writes a manifest you can inspect. For everyday small-to-medium dumps, mysqldump is still simpler and the recipes above are all you need; for anything where the backup window matters, it’s worth knowing mysqlsh exists.

Posted in Database, Linux, MySQL | Comments Off on MySQL Backup and Restore – Using Command Line