//Tips tagged sed
Latest tips by RSS
Click here to subscribe
Follow Shell-Fu on Twitter
Click here to follow
Follow Shell-Fu on identi.ca
Click here to follow
Converts the symbolic permissions to octal (ie: numbers) when using 'ls -l':

$ls -l | sed -e 's/--x/1/g' -e 's/-w-/2/g' -e 's/-wx/3/g' -e 's/r--/4/g' \
 -e 's/r-x/5/g' -e 's/rw-/6/g' -e 's/rwx/7/g' -e 's/---/0/g'

-755  1 jrl jrl  111943 2003-10-21 19:57 logscan
-644  1 jrl jrl   35468 2003-11-23 16:13 htfoo
-700  1 jrl jrl 3100672 2004-05-15 17:00 mutt
-644  1 jrl jrl   10162 2005-02-22 14:14 joinstep2.php
-777  1 jrl jrl   41079 2005-04-21 13:02 setistats
d755  2 jrl jrl      47 2007-10-26 14:41 rf
-700  1 jrl jrl     104 2008-02-05 11:26 getc
If you're going to use this, you may want to make an alias rather than type it in each time!


View Comments »


Add the following alias and function to your profile to be able to copy and paste files at the command line:

ccopy(){ cp $1 /tmp/ccopy.$1; }
alias cpaste="ls /tmp/ccopy* | sed 's|[^\.]*.\.||' | xargs -I % mv /tmp/ccopy.% ./%"

You can see below how this can be used:
blackbird:~/tst tks1$ ls
1.txt   2.txt   t1.tss  t2.tss  t3.tss
blackbird:~/tst tks1$ ccopy 1.txt
blackbird:~/tst tks1$ ccopy 2.txt

blackbird:~/tst tks1$ cd ../tst2 blackbird:~/tst2 tks1$ ls
blackbird:~/tst2 tks1$ cpaste
blackbird:~/tst2 tks1$ ls 1.txt 2.txt


View Comments »

YouTube is great, but the embedded flash player doesn't offer a lot in the way of customizing your viewing experience. There are quite a few options for downloading flv video files from YouTube, but adding the following function to your bashrc will let you stream them directly to your choice of media player.

function mtube {
  video_id=$(curl -s $1 | sed -n "/watch_fullscreen/s;.*\(video_id.\+\)&title.*;\1;p");
  mplayer -fs $(echo "http://youtube.com/get_video.php?$video_id");
}

This should work with any URL of the form http://www.youtube.com/watch?v=[video id here], for example:

mtube http://www.youtube.com/watch?v=evQ0QNi2CzA


View Comments »

Add commas to all numeric strings in a file, changing "1234567" to "1,234,567"

sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/;ta' filename



View Comments »

The following alias will print the directory structure from the current directory in tree format.

alias dirf='find . -type d | sed -e "s/[^-][^\/]*\//  |/g" -e "s/|\([^ ]\)/|-\1/"'


View Comments »

To create a set of backed up files with the current date added at the end of the file name try the following:

ls *txt | sed "s/.*/cp & &.$(date "+%Y%d%m")/"


This will run the following commands:

cp 1.txt 1.txt.20082703

cp 2.txt 2.txt.20082703
cp 3.txt 3.txt.20082703


View Comments »

Selective content replace on files. For example to replace '<?' with '<?php' in all PHP files:
find . -name '*.php' -exec sed -ie 's#<?#<?php#' {} \;


View Comments »

Mass-renaming files using find and sed:

find -name "*.php3" | sed 's/\(.*\).php3$/mv "&" "\1.php"/' | sh

(this example will rename all .php3 files to .php)


View Comments »

Prefer Perl over Sed? No problem! To use Perl as a Sed-like program:
perl -pe 's/foo/bar/; etc'

Have a Perl script ready? No problem!
perl -p foo.pl


View Comments »

Add the following sed commands to cal to get a calendar with the current date marked:

cal | sed "s/^/ /;s/$/ /;s/ $(date +%e) / $(date +%e | sed 's/./#/g') /"


View Comments »


Displays a random xkcd comic. Requires ImageMagick.

wget http://dynamic.xkcd.com/comic/random/ -O -| grep <img src="http://imgs.xkcd.com/comics | 
 sed s/<img src="// | sed s/"[a-z]*.*// | wget -i - -O -| display


View Comments »

Sometimes when using sed you find that you need to match across line endings, this can be achieved by getting sed to match the first line and then pulling a second line into the buffer with the N command.

For example, if we have a file:
$ cat foo
This is a sample hello
world file.

And want to change 'hello world' to 'hello shell-fu' we need to replace across lines. This can be done with the following command:
:~$ cat foo | sed '/hello$/N;s/hello\nworld/hello\nshell-fu/'
This is a sample hello
shell-fu file.

Here sed first looks for lines which end with 'hello' then reads the next line, finally replacing 'hello\nworld' with 'hello\nshell-fu'.

This also has a lot of other uses, for example converting double line spaced files to single:
cat doublespace | sed '/^$/N;s/\n$//g'


View Comments »

The following command creates in the /usr/project directory, a copy of the current working directory structure:

find . -type d -print|sed 's@^\.\{0,1\}@/usr/project@' | sed 's/ /\\ /' | xargs mkdir -p


View Comments »

Go to [http://www.bbc.co.uk/cgi-perl/weather/search/new_search.pl] and search for your location. Copy the link to the 3 day forecast feed, for example [http://feeds.bbc.co.uk/weather/feeds/rss/5day/world/0105.xml]. Then just put that link into the following command to get a quick command line weather forecast.

wget -q -O - http://feeds.bbc.co.uk/weather/.../XXXXX.xml | grep title | sed -e "s/<[^>]*>//g" -e "s/°//g" | egrep "^[A-Z]"

Example:
$ wget -q -O - http://feeds.bbc.co.uk/weather/feeds/rss/5day/world/0105.xml | grep title | sed -e "s/<[^>]*>//g" -e "s/°//g" | egrep "^[A-Z]"
BBC - Weather Centre - Forecast for San Francisco, United States of America Tuesday: sunny, Max Temp: 22C (72F), Min Temp: 13C (55F) Wednesday: sunny, Max Temp: 22C (72F), Min Temp: 15C (59F) Thursday: sunny, Max Temp: 25C (77F), Min Temp: 14C (57F)




View Comments »

Print the contents of a file from a given regular expression to another

sed -n '/start/,/end/ p' file

This will print the contents of the file from the line that matches /start/ until the line that matches /end/


View Comments »

Convert mac addresses such as 000000abde00 into 00:00:00:ab:de:00

awk '{for(i=10;i>=2;i-=2)$0=substr($0,1,i)":"substr($0,i+1);print}' macaddress_list
sed 's/\(..\)/\1:/g;s/:$//' macaddress_list

// sil at infiltrated.net


View Comments »

Mail somebody about space running low in some path (ksh, bash):


PATHS="/export/home /home"
AWK=/usr/bin/awk
DU="/usr/bin/du -ks"
GREP=/usr/bin/grep
DF="/usr/bin/df -k"
TR=/usr/bin/tr
SED=/usr/bin/sed
CAT=/usr/bin/cat
MAILFILE=/tmp/mailviews$$
MAILER=/bin/mailx
mailto="all@company.com"
for path in $PATHS
do
DISK_AVAIL=`$DF $path | $GREP -v "Filesystem" | $AWK '{print $5}'|$SED 's/%//g'`
if [ $DISK_AVAIL -gt 90 ];then
echo "Please clean up your stuff\n\n" > $MAILFILE
$CAT $MAILFILE | $MAILER -s "Clean up stuff" $mailto
fi
done


View Comments »

SSH_COMPLETE=( $(cat ~/.ssh/known_hosts | \
                 cut -f 1 -d ' ' | \
                 sed -e s/,.*//g | \
                 uniq | \
                 egrep -v [0123456789]) )
complete -o default -W "${SSH_COMPLETE[*]}" ssh


View Comments »

The following isn't particularly pretty and should be considered a work in progress, but it's quite fun.

Get examples of ways a command can be used direct from shell-fu by adding the following alias:
function examples { lynx -width=$COLUMNS -nonumbers -dump "http://www.shell-fu.org/lister.php?tag=$1" | \
sed -n '/^[a-zA-Z]/,$p' | egrep -v '^http|^javas|View Comm|HIDE|] \+|to Share|^ +\*|^ +[HV][a-z]* l|^ .*efu.*ep.*!$' | \
sed -e '/^  *__*/N;s/\n$//g' | less -r; }

This pulls out the tips tagged by the given command. (Make sure you tag any tips you submit!)


View Comments »

This bit of sed will print the contents of a file until the first line which doesn't contain the specified expression. A useful alternative to 'head' when you're not sure how much of the file you need.

sed -n '/Hello/!q; p'


View Comments »

Add a leading angle bracket and space to each line (quote a message)

sed 's/^/> /' filename


View Comments »

I'm working on a project to access data from my corporate Active Directory server using ldapsearch. The version of ldapsearch that comes with Red Hat Enterprise Server spews out LDIF in column truncated format. That is to say it inserts a carriage return at column 80 of the output.

This sed goodie unwraps the ldif output.

UNWRAP=' /^ / {; H; d; }; /^ /! {; x; s/\n //; }; '


View Comments »

Use the following command to give a history listing without the numbers for easier copy and pasting:

history | sed 's/^[ 0-9]* //'


View Comments »



Home Latest Browse Top 25 Random Hall Of Fame Contact Submit