Bash Mass Changes

Contents 1 Basic Tricks 1.1 Finding 1.2 Creating a List of Commands and Executing It 1.2.1 Escaping Hell 1.2.2 Escaping Purgatory 1.3 Constructing Command Line with Regular Expressions 2 Multi-Renaming 2.1 All Subdirs 2.2 One Level 3 Old Tips 3.1 File Editing 3.2 De-Danify Filenames 4 References

Basic Tricks

Finding

Print out a list of all regular files with full paths:

$ find -type f -printf "`pwd`/%P\n" 

Print non-svn files:

$ find -not -regex ".*\.svn.*"

Creating a List of Commands and Executing It

$ printf "echo hej\necho der\n"
echo hej
echo der
$ printf "echo hej\necho der\n" | xargs -i bash -c "{}"
hej                               \___________________/
der                                   |_ this is a magic spell causing
                                         every line of the input to be
                                         executed in turn

Escaping Hell

Print a command that prints a linefeed:

$ printf 'printf "\\\\n"\n'
printf "\\n"

Print a command that prints "hi":

$ printf "printf \\\"hi\\\"\n"
printf \"hi\"

A combination

$ printf "printf \\\"hi \\\"\nprintf \\\"there\\\\\\n\\\"\n"
printf \"hi \"
printf \"there\
\"
$ printf "printf \\\"hi \\\"\nprintf \\\"there\\\\\\n\\\"\n" | xargs -i bash -c "{}"
hi there

... probably one of the hardest ways of printing "hi there" ;)

Escaping Purgatory

Single quotes can be used instead for slightly better readability:

$ printf 'printf \"hi\"\n'
printf \"hi\"

$ printf 'printf \"\\\n\"\n'
printf "\\n"

$ printf 'printf \"hi \"\nprintf \"there\\\n\"\n'|xargs -i bash -c "{}"

Constructing Command Line with Regular Expressions

$ find -regex ".*\.flac$" -and -not -regex ".*\.flac\.flac" |        <- find specific files
  sed -re "s/(.*)/\1 \1/" |                                          <- duplicate file name
  sed -re "s/(.*) (.*)\.flac/flac -dc \1 | lame -b192 ->\2.mp3/g" |  <- construct command-line
                                                                        and replace extension
  xargs -i bash -c "echo {}"                                         <- execute each command
                      |
             remove this to activate

Multi-Renaming

The task in this section is renaming all files in a directory (and potentially all subdirectories). It is a bit tricky because filenames with space tend to ruin the command parsing.

All Subdirs

Check out this little beauty using find, awk, and xargs:

$ find | awk '/ / { new = gensub(/ /, "_", "g", $0);
  printf("mv -f \\\"%s\\\" \\\"%s\\\"\n", $0, new); }' | xargs -i bash -c "echo {}"

Here it is explained in greater detail:

$ find |
  awk '/ / {                                             -> only look at files with spaces (we use awk
                                                            for filtering instead of 'find' because awk
                                                            uses POSIX regexs)
      new = gensub(/ /, "_", "g", $0);                   -> replace space globally with underscore
      printf("mv -f \\\"%s\\\" \\\"%s\\\"\n", $0, new);  -> create command - the escaped backslash
                                                            and double quote will be correctly expanded
  }' | xargs -i bash -c "echo {}"                           by bash, when executed
                           |_ remove this to perform changes

One Level

For each file, print out 'mv filename filename_with_underscores'

$ for x in *.txt;do (printf "mv $x "|sed 's/ /\\ /g';printf "$x\n"|sed 's/ /_/g')|xargs -i bash -c "echo {}";done
                                                                         remove this to activate _____|

But 'for' is no good for using and 'find' with filenames with spaces:

$ find -printf '"%p"\n'
  "."
  "./one_word"
  "./two words"
$ for f in `find -printf '"%p"\n'`;do echo $f;done
  "."
  "./one_word"
  "./two              <- NOT good!
  words"

Old Tips

File Editing

remove all lines with '* file version :' in all files in this dir and subdirs

       $ for x in `find -name "*.*"`;do sed -e "/\*.*file version.*:/d" $x >$x.sed;mv $x.sed $x; done

remove all lines with '* TODO : ?' in all files in this dir and subdirs

       $ for x in `find -name "*.*"`;do sed -e "/\*.*TODO.*: \?/d" $x >$x.sed;mv $x.sed $x; done

find all files called '*CVS/Root' and change '3.80' to '3.80:' in them:

       $ for x in `find -regex ".*CVS/Root"`;do sed -e "s/3\.80/3\.80:/" $x >$x.sed;mv $x.sed $x; done

change all date signatures in all files (except CVS files) from old dd/mm/yy to ISO 8601 format

(ccyy/mm/dd):

       $ for x in `find -not -regex ".*CVS.*"`; do sed -ri -e "s/([0-9]{2})\/([0-9]{2})\/([0-9]{2})/20\3-\2-\1/g" $x;done

if it is done in a CVS checkout, check the changes with this

       $ cvs di|less

De-Danify Filenames

I want to rename all files and directories with name containing the danish letters 'æøåÆØÅ' to corresponding files/directories without these characters. The problem with this task is that the directory must be traversed one level at a time (breadth-first) to avoid accessing invalid directory names. This makes it necessary to use a script:

LETTERS="[æøåÆØÅ]+" MAXDEPTH=255

d=1;while [ $d -lt $MAXDEPTH ];do

   printf "processing files at level %d... " $d;

   # do any files still need to be moved?
   if find -mindepth $d -name "*"|egrep $LETTERS>/dev/null;then

       # list all files with danish letters in this directory tree level
       find -mindepth $d -maxdepth $d -name "*"|egrep $LETTERS >/tmp/rename.tmp;

       # make a new list with all the danish letters changed
       sed -e "s/æ/ae/g" -e "s/ø/oe/g" -e "s/å/aa/g" -e "s/Æ/AE/g" -e "s/Ø/OE/g" -e "s/Å/AA/g" </tmp/rename.tmp >/tmp/rename2.tmp

       # move the files
       paste /tmp/rename.tmp /tmp/rename2.tmp|xargs -n2 mv

       d=$((d+1));
   else d=$MAXDEPTH; # we are done with this level
   fi;
   printf "done.\n";

done

References

• http://www.delorie.com/gnu/docs/gawk/gawk_135.html