Search code examples
.netbatch-filecopycmdhidden-characters

Removing hidden file end characters (BOM) in visual studio


I'm using a batch file to concatenate all my css files

copy /b reset.css+action-plan.css+buttons.css+behaviours.css+footer.css+forms.css+header.css+home.css+layout.css+lightbox.css+print.css+questionnaire.css+typography.css+you-told-us.css main.css

I've done this numerous times before on various projects, but this project uses .NET and the files are all being edited in visual studio.

The problem I have is that there are some mysterious hidden characters being added at the end of each file, which, when concatenated, causes the resulting css to be invalid.

 126  BLOCKQUOTE, Q   Lexical error at line 119, column 1.
 Encountered: "?" (63), after : "" ??? /**** left column ****/ 

All the individual CSS files validate and the errors are only thrown in the combined file at teh points were the individual files join.


Solution

  • The problem is because of the byte order mark (BOM) in your files. The byte order mark is for unicode files to tell the processor the order of the bytes. You can read more about it here:

    http://en.wikipedia.org/wiki/Byte_order_mark

    The problem is that Visual Studio is adding those marks to your css file and when you combine them by concatenation, BOMs are ending up in the middle of the text, screwing things up.

    When you go to the Save As dialog, you can expand the Save button to see the 'Save with Encoding' option. This will prompt you for a different encoding, and I think one of the Unicode options will leave out the BOM (somewhere in the list is UTF-8 without signature).

    I don't know how to set Visual Studio to use a specific encoding by default.

    To skirt the issues I created a program to concatenate files that would respect the BOM. I use that rather than copy, or the unix cat.