find UTF-8 files with BOM (remove BOM)

http://stackoverflow.com/questions/204765/elegant-way-to-search-for-utf-8-files-with-bom

find -type f|while read file;do [ "`head -c3 -- "$file"`" == $'xefxbbxbf' ] && echo "found BOM in: $file";done
grep -orHbm1 "^`echo -ne 'xefxbbxbf'`" . | sed '/:0:/!d;s/:0:.*//'

most efficient way:
find . -type f -print0 | xargs -0r awk '/^xEFxBBxBF/ {print FILENAME} {nextfile}'

find and remove BOM:
find . -type f -exec sed -i 's/^xEFxBBxBF//' {} ;

just remove BOM:
tail –bytes=+4 text.txt

Leave a Reply

Your email address will not be published. Required fields are marked *