An apostrophe is used for resolving ambiguity. It is used in the following cases.
Both Method I and Method II:
When ㅇ appears at the initial position of a non-initial syllable of a polysyllabic word (e.g. 앉아라anc'ara/anj'ara)
When ㄲ, ㄸ, ㅃ, ㅉ, or ㅆ appears at the initial position of a non-initial syllable of a polysyllabic word (e.g. 아까a'kka/a'gga)
Method I only:
When ㅋ, ㅌ, ㅍ, or ㅊ appears at the initial position of a non-initial syllable of a polysyllabic word (e.g. 애타다ae'thata)
In fact, the above rules are always applied; they are applied even when there is no ambiguity (e.g. 아이a'i, 흰떡hyin'tteok/hyin'ddeog).
In character names in Unicode and ISO/IEC 10646
In Unicode and ISO/IEC 10646, both Method I and Method II are used for character names.
Method I: characters in the Hangul Jamo and Hangul Compatibility Jamo blocks (e.g. U+1103 ᄃHANGUL CHOSEONG TIKEUT (디귿), U+3148 ㅈHANGUL LETTER CIEUC (지읒))
Method II: characters in the Hangul Syllables block (e.g. U+AF43 꽃HANGUL SYLLABLE GGOC)
As an apostrophe is not allowed in character names, apostrophes are always omitted. For example, 옛이응 is yes'i'eung in ISO/TR 11941, but YESIEUNG in Unicode and ISO/IEC 10646 (e.g. U+114C ᅌHANGUL CHOSEONG YESIEUNG).
Notes
^Not transliterated word-initially, ' (apostrophe) as a syllabic initial (always, except word-initially; e.g. 아이a'i), ng as a syllabic final (e.g. 방pang/bang).
^r as a syllabic initial, l as a syllabic final (e.g. 를reul).