alexharv074.github.io

My blog

View on GitHub
2 April 2019

My sed and AWK Cookbook

by Alex Harvey

This is a list of my favourite productivity-enhancing sed & AWK one-liners.1

Edit files in place

You want to replace all instances of a pattern in a file in place without saving a backup.

sed -i '' 's/SEARCH/REPLACE/g'
gsed -i 's/SEARCH/REPLACE/g' FILE

Inserting lines before or after patterns

After each instance of a pattern

You want to insert a line foo after a pattern PATTERN in a file.

gsed -i '/PATTERN/a foo' FILE

If you need to also insert say 2 leading newlines:

gsed -i -e '/PATTERN/a\' -e '  foo' FILE

Before each instance of a pattern

You want to insert a line foo before a pattern PATTERN in a file.

gsed -i '/PATTERN/i foo' FILE

After the last instance of a pattern

You want to insert a line foo after the last instance of a pattern PATTERN in a file.

sed -i '1h; 1!H; $!d; x; s/.*PATTERN[^\n]*/&\nfoo/' FILE

See Stack Overflow.

For N=1:

gsed -n '/PATTERN/{n;p}' FILE

For N=2:

gsed -n '/PATTERN/{n;n;p}' FILE

For N=1:

sed '$!N; /.*\n.*PATTERN/P; D' FILE

For N=2:

sed '1N; $!N; /.*\n.*\n.*PATTERN/P; D' FILE

For N=3:

sed '1{N;N};$!N;/.*\n.*\n.*\n.*pattern/P;D'

For N=4:

sed '1{N;N;N}; $!N; /.*\n.*\n.*\n.*\n.*PATTERN/P; D' FILE

See Stack Overflow.

For example, the 11th line of a file:

sed -n 11p

The 4th to 11th lines inclusive in a file:

sed -n 4,11p

The 4th line to the end of file:

sed -n '4,$p'

Suppose you have these lines in a file:

aaa
PATTERN1
bbb
ccc
PATTERN2
ddd
PATTERN1
eee
fff
PATTERN2
ggg
PATTERN1
hhh

You want to return these lines:

PATTERN1
bbb
ccc
PATTERN2
PATTERN1
eee
fff
PATTERN2
PATTERN1
hhh
sed -n '/PATTERN1/,/PATTERN2/p' FILE
awk '/PATTERN1/,/PATTERN2/' FILE

On Stack Overflow here and here.

Suppose you only want to return these lines:

PATTERN1
bbb
ccc
PATTERN2
sed -n '/PATTERN1/,/PATTERN2/p;/PATTERN2/q'
awk '/PATTERN1/,/PATTERN2/;/PATTERN2/{exit}' FILE

Suppose you only want to return these lines:

bbb
ccc
eee
fff
gsed -n '/PATTERN1/,/PATTERN2/{//!p}' FILE
awk '/PATTERN1/,/PATTERN2/{if(/PATTERN2|PATTERN1/)next;print}' FILE

Suppose you only want to return these lines:

bbb
ccc
gsed '0,/PATTERN1/d;/PATTERN2/Q' FILE
awk '/PATTERN1/{f=1;next}/PATTERN2/{exit}f' FILE

On Stack Overflow here and here.

Remove trailing whitespaces in a file

gsed -i 's/  *$//' FILE

Search for a pattern within a function or block

Problem you want to search or “grep” for a pattern, but only within a function or block of code.

Suppose you have some text:

aaa
bbb
function() {
  ccc
  ddd
  PATTERN
  eee
  fff
}
ggg
PATTERN
hhh
sed -n '/^function/,/^}/{/PATTERN/p;}' FILE

1 Note that many of these one-liners are auto-generated from unit test code.

tags: sed - awk