Nailing the colours (also nailing the colours to the mast or nailing the flag) is a practice dating back to the Age of Sail that expresses a defiant refusal to surrender, and willingness to fight to the last man.
During the Age of Sail, ships would legally fight only while flying their national flag. Flying another flag was considered to be a legitimate ruse de guerre only until the beginning of the fight. Striking the colours was a sign of surrender. Indeed, when shot or shrapnel felled a ship's flag (such as by severing the halyard that held it up), her opponent would cease firing and inquire whether she was capitulating.
In contrast, fixing the battle ensign with nails would prevent it from being removed easily, and effectively prevented the surrender. It became an expression of defiance and willingness to force oneself to fight up to the bitter end.