Ronald Pringadi's Technical Notes

Category: Operating System

  • Search and replace withing a file


    sed -i -e"s/SEARCH_TEXT/REPPLACE_TEXT/g" FILENAME.txt

    # another example using bakup
    sed -i.bak -e"s/SEARCH_TEXT/REPPLACE_TEXT/g" FILENAME.txt

    # another example using variable
    sed -i -e"s/\/pathname/\/pathname-$date/g" "filename-"$date".xml"

  • Bash Scripting Basics (Variable Assignment, If, Loop)


    #!/bin/bash
    ##################################################
    # Benchmark the processing time when wkhtmltopdf
    # converts an html file to a pdf file
    ##################################################

    if [[ $1 = "" || $2 = "" || $3 = "" ]]; then
    echo "Usage: `basename $0` sourceFile.html destinationFile.pdf iteartionRunTime"
    else
    _min=9999
    _max=0
    _total=0
    _iteration=$3
    for (( _i=1; _i<=$_iteration; _i++ )) do _start_time=`date +%s` echo "running $_i" ~MYUSER/bin/benchmarkWkhtmltopdf.sh $1 $2 > /dev/null 2>&1
    _end_time=`date +%s`
    _processing_time=$((_end_time-_start_time))
    if [ "$_processing_time" -lt "$_min" ]; then
    _min=$_processing_time
    fi
    if [ "$_processing_time" -gt "$_max" ]; then
    _max=$_processing_time
    fi
    _total=$(($_total+$_processing_time))
    done
    _processing_time_average=$(echo - | awk "{print $_total/$_iteration}")
    echo "Processing time avg: $_processing_time_average min:$_min max:$_max total:$_total"
    fi

  • Compress and Extract Files in Linux

    Extract:

    .tar.bz2
    To extract a tar (Tape ARchiver) file, type the following in the shell prompt:

    tar xvjf yourFileName.tar.bz2

    Which will untar it to the current directory. Depending on the file structure that being compressed, it might create sub-directories.
    Parameters:
    x – extract
    v – verbose output (lists all files as they are extracted)
    j – deal with bzipped file
    f – read from a file, rather than a tape device

    .tar.gz or .tgz

    tar xvzf file.tar.gz
    tar xvzf file.tgz

    Parameters:
    -x, –extract, –get extract files from an archive
    -v, –verbose verbosely list files processed
    -z, –gzip, –gunzip, –ungzip filter the archive through gzip
    -f, –file=ARCHIVE use archive file or device ARCHIVE

    .bz2 file

    tar jxf YOUR_FILE_NAME.bz2

    .zip file

    unzip YOUR_FILE_NAME.zip

  • Remote copy using scp

    Do you ever need to copy one file from one server to another? If you have ssh access to the remote server then you can do an scp command like so:


    $ scp remoteUsername@remoteServername:remoteFile localFile

  • Delete files older than X number of days in Linux

    Say you backup your database daily and you need to delete the old backup files.
    You can issue the command bellow which will erase any files within the specified directory that are older than 7 days.


    $ find /your/target/directory* -mtime +7 -exec rm {} \;

  • Calling a mysql query from linux or dos command line

    There are times when you want to call a mysql command from the bash or dos script then call this script in a scheduler (cron job).
    The example below will show you how to do so.

    mysql -uYOURUSERNAME -pYOURPASSWORD -DYOURDATABASE -e"CALL YOURSTOREDPROCEDURE('YOUR_SP_PARAMETER')"

    Explanation:
    -u is where you put your username, you can either put a space or no space at all after the -u (both works). As you can see I prefer not to put any space.
    -p is where you put your password.
    -D is your database name. That’s a capital D.
    -e this is where your query will go. It needs to be quoted (“). If it’s calling a stored procedure or stored function, you’ll need to use the CALL keyword, otherwise if it’s just a simple query don use the CALL keyword.

  • Remove all svn directory

    Subversion (SVN) sometimes clutter our source directory by creating .svn folder in each folder or sub folder that we version control.

    The easiest way to erase them is by using a bash command:


    find . -name ".svn" -type d -exec rm -rf {} \;

  • Adding New VirtualHost in Apache2

    On linux Ubuntu
    1. If your apache has /etc/apache2/sites-available directory then just create a new file “dev.newwebsite.com.conf”
    with the following content:


    ServerName dev.newwebsite.com
    DocumentRoot "/var/www/www.newwebsite.com"

    # Error handlers
    ErrorDocument 500 /errordocs/500.html


    AllowOverride All
    Options -Indexes FollowSymLinks
    Allow from all


    2. Also copy the same file into /etc/apache2/sites-enabled

    3. Restart your apache

    $ /etc/init.d/apach2 restart

    4. Edit your hosts file: /etc/hosts
    add the following line to it
    127.0.0.1 dev.newwebsite.com

    5. Open your web browser and open http://dev.newwebsite.com

    If you are a windows user and are using xampp then you need to alter the vhost configuration file.
    The default xampp file is in
    “\xampp\apache\conf\extra\httpd-vhosts”
    Then restart your apache

  • SVN Tricks in Linux

    To show colorized svn diff


    sudo aptitude install colordiff

    Then on your home directory there is a hidden file called .bashrc.
    Edit it.


    vim ~/.bashrc

    Append the following code at the end of the file

    svndiff()
    {
    svn diff "${@}" | colordiff
    }

    Reload the .bashrc file:

    ~/.bashrc

    Next to view the diff use svndiff instead of svn diff

    Removing all .svn subfolders


    rm -rf `find . -type d -name .svn`

    SVN Revert All


    svn st -q | awk '{print $2;}' | xargs svn revert

  • Robocopy – Automate Your Windows Copy

    I backup my files regularly using the windows scheduler. For example several folders from my windows preferences folder in drive C (C:\Users\dellxps\AppData\Roaming) to another drive, to D:\Users\Ronald\AppData\Roaming.

    There is a built in smart copier in windows that is not known by many people called robocopy.
    Take a look at the following scripts: backup.bat and restore.bat

    backup.bat

    @echo off
    SET vSourceRootFolder=C:\Users\dellxps\AppData\Roaming
    SET vDestinationRootFolder=D:\Users\Ronald\AppData\Roaming
    echo "========================================"
    echo "== Backup application preference files"
    echo "========================================"
    robocopy %vSourceRootFolder%\\"FileZilla\\" %vDestinationRootFolder%\\"FileZilla\\" /MIR /R:2 /W:0 /XJD /XJ /NP /LOG+:D:\sync\backup_d_drive.log
    robocopy %vSourceRootFolder%\\"Mozilla\\" %vDestinationRootFolder%\\"Mozilla\\" /MIR /R:2 /W:0 /XJD /XJ /NP /LOG+:D:\sync\backup_d_drive.log

    restore.bat

    @echo off
    SET vDestinationRootFolder=C:\Users\dellxps\AppData\Roaming
    SET vSourceRootFolder=D:\Users\Ronald\AppData\Roaming
    echo "========================================"
    echo "== Backup application preference files"
    echo "========================================"
    robocopy %vSourceRootFolder%\\"FileZilla\\" %vDestinationRootFolder%\\"FileZilla\\" /MIR /R:2 /W:0 /XJD /XJ /NP /LOG+:D:\sync\backup_d_drive.log
    robocopy %vSourceRootFolder%\\"Mozilla\\" %vDestinationRootFolder%\\"Mozilla\\" /MIR /R:2 /W:0 /XJD /XJ /NP /LOG+:D:\sync\backup_d_drive.log

    And here is the documentation from robocopy /?

    -------------------------------------------------------------------------------
    ROBOCOPY :: Robust File Copy for Windows
    -------------------------------------------------------------------------------

    Started : Wed Sep 19 21:48:29 2012

    Usage :: ROBOCOPY source destination [file [file]...] [options]

    source :: Source Directory (drive:\path or \\server\share\path).
    destination :: Destination Dir (drive:\path or \\server\share\path).
    file :: File(s) to copy (names/wildcards: default is "*.*").

    ::
    :: Copy options :
    ::
    /S :: copy Subdirectories, but not empty ones.
    /E :: copy subdirectories, including Empty ones.
    /LEV:n :: only copy the top n LEVels of the source directory tree.

    /Z :: copy files in restartable mode.
    /B :: copy files in Backup mode.
    /ZB :: use restartable mode; if access denied use Backup mode.
    /EFSRAW :: copy all encrypted files in EFS RAW mode.

    /COPY:copyflag[s] :: what to COPY for files (default is /COPY:DAT).
    (copyflags : D=Data, A=Attributes, T=Timestamps).
    (S=Security=NTFS ACLs, O=Owner info, U=aUditing info).

    /DCOPY:T :: COPY Directory Timestamps.

    /SEC :: copy files with SECurity (equivalent to /COPY:DATS).
    /COPYALL :: COPY ALL file info (equivalent to /COPY:DATSOU).
    /NOCOPY :: COPY NO file info (useful with /PURGE).

    /SECFIX :: FIX file SECurity on all files, even skipped files.
    /TIMFIX :: FIX file TIMes on all files, even skipped files.

    /PURGE :: delete dest files/dirs that no longer exist in source.
    /MIR :: MIRror a directory tree (equivalent to /E plus /PURGE).

    /MOV :: MOVe files (delete from source after copying).
    /MOVE :: MOVE files AND dirs (delete from source after copying).

    /A+:[RASHCNET] :: add the given Attributes to copied files.
    /A-:[RASHCNET] :: remove the given Attributes from copied files.

    /CREATE :: CREATE directory tree and zero-length files only.
    /FAT :: create destination files using 8.3 FAT file names only.
    /256 :: turn off very long path (> 256 characters) support.

    /MON:n :: MONitor source; run again when more than n changes seen.
    /MOT:m :: MOnitor source; run again in m minutes Time, if changed.

    /RH:hhmm-hhmm :: Run Hours - times when new copies may be started.
    /PF :: check run hours on a Per File (not per pass) basis.

    /IPG:n :: Inter-Packet Gap (ms), to free bandwidth on slow lines.

    /SL :: copy symbolic links versus the target.

    /MT[:n] :: Do multi-threaded copies with n threads (default 8).
    n must be at least 1 and not greater than 128.
    This option is incompatible with the /IPG and /EFSRAW options.
    Redirect output using /LOG option for better performance.

    ::
    :: File Selection Options :
    ::
    /A :: copy only files with the Archive attribute set.
    /M :: copy only files with the Archive attribute and reset it.
    /IA:[RASHCNETO] :: Include only files with any of the given Attributes set.
    /XA:[RASHCNETO] :: eXclude files with any of the given Attributes set.

    /XF file [file]... :: eXclude Files matching given names/paths/wildcards.
    /XD dirs [dirs]... :: eXclude Directories matching given names/paths.

    /XC :: eXclude Changed files.
    /XN :: eXclude Newer files.
    /XO :: eXclude Older files.
    /XX :: eXclude eXtra files and directories.
    /XL :: eXclude Lonely files and directories.
    /IS :: Include Same files.
    /IT :: Include Tweaked files.

    /MAX:n :: MAXimum file size - exclude files bigger than n bytes.
    /MIN:n :: MINimum file size - exclude files smaller than n bytes.

    /MAXAGE:n :: MAXimum file AGE - exclude files older than n days/date.
    /MINAGE:n :: MINimum file AGE - exclude files newer than n days/date.
    /MAXLAD:n :: MAXimum Last Access Date - exclude files unused since n.
    /MINLAD:n :: MINimum Last Access Date - exclude files used since n.
    (If n < 1900 then n = n days, else n = YYYYMMDD date). /XJ :: eXclude Junction points. (normally included by default). /FFT :: assume FAT File Times (2-second granularity). /DST :: compensate for one-hour DST time differences. /XJD :: eXclude Junction points for Directories. /XJF :: eXclude Junction points for Files. :: :: Retry Options : :: /R:n :: number of Retries on failed copies: default 1 million. /W:n :: Wait time between retries: default is 30 seconds. /REG :: Save /R:n and /W:n in the Registry as default settings. /TBD :: wait for sharenames To Be Defined (retry error 67). :: :: Logging Options : :: /L :: List only - don't copy, timestamp or delete any files. /X :: report all eXtra files, not just those selected. /V :: produce Verbose output, showing skipped files. /TS :: include source file Time Stamps in the output. /FP :: include Full Pathname of files in the output. /BYTES :: Print sizes as bytes. /NS :: No Size - don't log file sizes. /NC :: No Class - don't log file classes. /NFL :: No File List - don't log file names. /NDL :: No Directory List - don't log directory names. /NP :: No Progress - don't display percentage copied. /ETA :: show Estimated Time of Arrival of copied files. /LOG:file :: output status to LOG file (overwrite existing log). /LOG+:file :: output status to LOG file (append to existing log). /UNILOG:file :: output status to LOG file as UNICODE (overwrite existing log). /UNILOG+:file :: output status to LOG file as UNICODE (append to existing log). /TEE :: output to console window, as well as the log file. /NJH :: No Job Header. /NJS :: No Job Summary. /UNICODE :: output status as UNICODE. :: :: Job Options : :: /JOB:jobname :: take parameters from the named JOB file. /SAVE:jobname :: SAVE parameters to the named job file /QUIT :: QUIT after processing command line (to view parameters). /NOSD :: NO Source Directory is specified. /NODD :: NO Destination Directory is specified. /IF :: Include the following Files.