XCD Specifications: initial draft (29-Apr-2002) ----------------------------------------------- discussed by de_xt, koepi and avih actual document by avih General: -------- The XCD is a CD container format standard for multimedia content (much like VCD or SVCD). In contrast to S/VCD though, it allows arbitrary multimedia content TYPES (VCD/SVCD supports only MPEG1/MPEG2 system streams respectively and images). XCD can support, for instance, MPEG4, VP3, DIVX, XVID, JPG, etc.. ANY media type. It uses reduced Error Correction facilities, such that more space of the CD can be used for actual media (like S/VCD, compared with current computer CDs). Due to the reduced error correction, it's aimed at stream/media types that can handle errors to some extent at least. Yet, it allows keeping arbitrary 'special' sections of the stream/media 'protected' (i.e. special headers/indexes etc). It contains recommendations for the player as for the order of the streams to play (in the form of optional playlists). Targets: -------- - Allowing arbitrary stream/media containers/types (i.e. ogg, mcf, avi, mp3, wav, jpeg, gif etc). - To be playable by both stand alone players and computer players. - Keep the minimal player implementation requirements very low for the XCD overhead. (Naturally, it will also have to include actual decoders for supported stream/media containers/types). - Offering optional advanced features such as playlists, Interactive navigation (similar to CD-i) and more for players that support such features, and arbitrary binary information (files) to be used by computer users. Annotations: ------------ XCD = Extended Compact Disc EDC = Error Detection Code (i.e. CRC - allows to check if errors exist, but not to recover them) ECC = Error Correction Code (i.e. Some redundency to allow reconstruction of data which is [to some extent] damaged) mode1 = CD format: ISO filesystem for binary content (as used by todays computer CDs). Error Handling: EDC and ECC. m2f1 = CD format: Mode 2 Form 1 iso 9660 standard. Error handling: EDC and ECC. m2f2 = CD format: Mode 2 Form 2 iso 9660 standard. Error handling: EDC. Bridge= Special CD mode that allows both m2f1 and m2f2 files to exist on one CD (like S/VCD) TBD = To Be Defined = Stream Serial Number. Four numeric characters in ASCII format. Padded with leading '0' if necessary. = Playlist Serial Number. Same format as . 1. CD Layout 1.1 the CD is of Bridge CD-ROM XA Format, with both form 1 and form 2 files. 1.2 All the files comply with 8.3 naming convention. All caps. (Not including [BIN] content??) 1.3 The directory structure is 'flat'. i.e. no nested directories (Except for [BIN]???). 1.4 Each stream/media is to be identified by 4 numeric characters (Stream Serial Number aka ) Thus allowing up to 10,000 streams/medias on a single XCD CD. ( is in decimal format). The should be sequential (with no 'gaps') starting with 0000 and possibly up to 9999. 1.5 Directory structure: [XCD] - The bridge(s) to the file(s) saved as m2f2 track(s) ????(TBD) [HEADERS] - Extra streams/Medias info (as the original file name, media tags etc) and backup of the vital streams content (i.e. headers/indexes/etc) Naming convention: HEAD.XCH i.e. 'HEAD0007.XCH' [MEDIA] - Actual streams/media files as m2f2 files. Naming convention: STRM.XCS i.e. 'STRM0007.XCS' [PLAYLIST]- Predefined playlists for the content of the CD. Naming convention: PLAY.XCL i.e. 'PLAY0002.XCL' [BIN] - Arbitrary binary content for computer users. Could be used for Players, filters etc. [XCDI] - Place for interactive stuff like on CDI (TBD) 2. Files content 2.1 [XCD] - TBD 2.2 [HEADERS] - each HEAD.XCH file referes to a specific single STRM.XCS file. each XCH file is in m2f1 and has the following structure: char xcdDescriptor[64??] // TBD- i.e. "XCD HEADER V1.0" byte protectionScheme; // 0=default protection scheme, 1=predefined scheme, 2=manual scheme. char originalFileName[256]; // NULL terminated ASCII string. UINT64 streamSize; // BIG/SMALL ENDIAN?? (what's the x86 format??) TAG mediaTag[2048??]; // TBD- ID3, or other media tag convention. UINT64 numBackedupSectors // number of backed up sector that this header contains. {XCD_BAK}+ // actual backups for sectors of STRM.XCS each XCD_BAK section has the following structure: typedef struct cxd_bak { UINT64 sectorNumber; // Referes to the sector beeing backed up from STRM.XLS // the 1st sector of STRM.XLS would be 0 for that matter. UINT64 dataOffset; // Data offset in bytes, from the begining of the original file. // Applied AFTER stripping the headers and edc (or others) associated with this media. // This is redundant information to make life easier for the parser. char userData[2324] // Actual backup of the user data section of the backed up sector. } XCD_BAK Example of such complete header, named 'HEAD0008.XCH': offset(dec)| content -----------|------------------ | "XCD HEADER V1.0" // i added the descriptor and protection scheme after i finished the table... :( // so the absolute offsets are incorrect | 1 // predefined approved protection scheme. 0 | TheMatrix.ogg\0 ... // padded with trailing useless chars 256 | 780,000,000 // original file size as UINT64 (ENDIAN??) 320 | "blah blah blah" // tag that give auxiliary info about the stream. TBD 2368 | 2 // 2 as UINT64 (num of backed up sectors) 2432 | 0 // back up for sector #0: sector number as UINT64 2496 | 0 // (redundant) data offset of original data to be backed up 2560 | // actual backup: block of data, sized 2324 bytes 4884 | 1 // backup for sector #1. 4948 | 2324 // (redundant) data offset of original data 5012 | // actual backup for sector #1 7336 | the above header relates to the media stream 'STRM0008.XCS'. it contains: - The original name of the file ('TheMatrix.ogg') - The original size of the file (~780M) - Some extra information regarding this clip. - Backups for 2 sectors: sector#0 and sector#1 of the 'STRM0008.XCS' m2f2 file. 2.3 [MEDIA] - each STRM.XCS is a standard m2f2 file generated from the original media file, padded with trailing '0' if the size is not a multiple of 2324 (to match the standard sector size) 2.4 [PLAYLIST] - each PLAY.XCL is a m2f1 file, and contain a list of streams to play one after another. it may refenence other playlists, or notify to change the media (insert the next CD for example). it may offer a specific timestamp to jump to in the specified stream. it may indicate a delay to issue AFTER the item has been played. it MUST NOT be recursive with relation to other playlists on the CD. The structure of XCL file is as follows: the 1st line will be an xcd identifier i.e. "XCD PLAYLIST V1.0" typedef struct play_list_element { char itemType; // can be: 'S' for a stream on this cd, 'P' for another playlist or 'M' for a media change. PLAY_ITEM item; } PLAY_LIST_ELEMENT; typedef struct stream_pointer { UINT32 ssn; // UINT32 timestamp // in msec, i.e. 1000 for 1 second. UINT32 delayAfter // in msec } STREAM_POINTER; typedef struct playlist_pointer { UINT32 psn; // UINT32 delayAfter // in msec } PLAYLIST_POINTER; typedef struct media_pointer { UINT32 mediaNumber; // i.e. 3. will show a message "Please insert CD #3" } MEDIA_POINTER; typedef union play_item { STREAM_POINTER stream; PLAYLIST_POINTER playList; MEDIA_POINTER media; } PLAY_ITEM; *** another way to implement this can be as an ascii file, instead of binary, so it can be edited using standard ascii editor. feedbacks regarding you preference is welcome. 2.5 [BIN] - this directory contains arbitraty binary files, to be used by computer users. i.e. players, filters etc. 2.6 [XCDI] - TBD- this directory contains interactive stuff. like menus. may support 'overlay' of the menu on top of a specific stream or playlist. if you know of a free good menu system, let me know. otherwise, we'll invent one :) 3. Minimal Requirements 3.1 Mastering package minimal requirements: - create the correct cd format (bridg xa) and the [XCD] directory content. - create the correct directory structure (all directories are compulsury, even if empty) - be able to utilize protection schemes (regular expressions?) for different file types. - create at least one STRM.XCS file with matching HEAD.XCH file, acording sections 2.2 and 2.3 - minimal protection scheme (REGARDLESS of downloadable/manual/etc): the 1st sector MUST be protected. 3.2 Player minimal requirements: - be able to read the cd content (xa m2f1 and m2f2) - be able to associate header file with a matching stream file, and utilize the protections implemented in the header (such they are regarded together as a single COMPOUND_STREAM) - be able to identify (either by content or file extension) if it can parse the COMPOUND_STREAM - it's most recommended that it has at least decoders for a single media stream type. - fail 'gracefully' if the content cannot be rendered (i.e. if an ogg stream contain xvid video and ac3 audio, but the player doesn't have ac3 decoder, it should at least play the video, (assumming it has xvid decoder of course)) 4. Opional features: 4.1 Mastering package optional features: - creation of correct playlists - adding binary files to the [BIN] directory - create interactive menus. 4.2 Player optional features: - read and 'render' playlists correctly - render menus (either with or without 'content overlay') 4.3 note: To support the XCD format, the minimal requirements of the player is to be able to read and parse the [HEADERS] and [STREAMS] dirs. (Note however, that not all players will support all formats. i.e. some players may not support avi clips, while supporting mcf and ogg clips. portable stand alone players may support only mp3 and ogg streams if they wish, or support avi files, but render the audio part only, etc.) 5. protection schemes for specific stream types. we should aim at a repository of protection schemes (regular expressions), that allaow each file type to have different protected sectors. the default and minimal protection scheme is to protect the 1st sector of each stream. that might be enough for many stream types. ------------- end of initial spec -------------- additional thoughts 30-APR-2002 -------------------------- additional [XTRAHEAD] and [XTRASTRM] for extra content besides the 'standard experience' content. i.e. clips for menu overlays, interviews etc. player support: XCD Level 0 - 1. identify the CD as an XCD, and able to read the directory structure. 2. able to read the header files of the standard and extra streams (for original name and filesize). 3. able to enumerate the streams (??) XCD Level 1 - 1. the player has built in decoder and renderer for at least one stream/media type. 2. enable the user to play each stream individually, either standard or exta. 3. enable the user to seek th stream, if the stream type allows it. XCD Level 2 - 1. able to play all of the 'standard' streams sequencially in one go (=standard experience) (only for streams that the player has decoder/s for) 2. full support for XCD playlists. XCD Level 3 - 1. support for XCD interactive menus. (supporting overlayed menus is optional, but the menu files must be parsed correctly anyway) Example: The new HellPlayer042 is fully XCD Level 1 compliant. It supports MP3 and Ogg streams. HellPlayer042 supports MP3 streams with or without CRC. HellPlayer042 supports Ogg streams with video of types DIVX4.xx/XVID/VP3, and/or Audio of types MP3/vorbis. ---------------------------------------------------------------------------------------------------- 22-May-2002 'CD Level XCD' and 'File Level XCD' ----------------------------------- To make development and implementation faster, we decided to divide the spec into 2 parts: CD Level XCD (aka CLX) ---------------------- This spec is the 'complete' spec of XCD, as related to in the draft proposal above. CLX is currently under development. File Level XCD (FLX) -------------------- This is a 'loose' spec for XCD, on a file level only. Which means, only relates to single media playback. it does not have a hardcoded directory structure, hardcoded naming conventions for the files, nor does it have playlists and menus (at least atm). FLX is defined in order to make advancement in one of the core features of XCD: usage of mode2 form 2 files with backup sections in mode2 form 1. It's intended for computer playback (although stand alone players may support it as well), and we consider FLX as a transition stage untill CLX is properly defined. Features of FLX: ---------------- - M2f2 media file and m2f1 Backup file fully supported (That's the main aim for FLX) . - Any file name is accepted (i.e. no restriction with 8.3 convention). - The file can reside in any directory (and not only in \'media' directory) - No hardcoded directory structure. - No hardcoded sector or tracks layout. - File access assumes OS level file access. FLX Specifications V1.0 (22-May-2002) ------------------------------------- FLX only defines the structure of the mode 2 form 1 header file, and it's naming and directory relation to the actual mode 2 form 2 media file. 1. FLX naming conventions: -------------------------- - The m2f2 media file should be with a 'dat' extension. - The header(=backup) file should have the same name as the media file, but with 'xch' extension. - The header file can be in the same directory as the media file, or in directory called 'secure' relative to the directory of the media file. example: - the original media file name is 'TheMatrix.avi' - When converting it to mode2 storage, the new file will be \'TheMatrix.dat' (where can be any directory, including the root directory). - The header file (mode 2 form 1) will be named 'TheMatrix.xch', and can reside either in or in \secure. No directory other than those 2 is supported under FLX. 2. Header file: --------------- 2.1 string and numeric conventions in the header file: - Each textual value is NULL terminated ascii string. - Each numeric value is 8 bytes, _int64 (Littele endian, as in x86 architecture) 2.2 header structure (in the form of <#bytes-field_name>): <32-(text)-XCD_Header_Descriptor> <256-(text)-Original_Filename> <8-Original_file_size> <8-Backup_section_offset> here we may add more fields in the future. the Backup section: <8-number_of_backup_segments> and then, for each segment: <8-seg_size> <8-seg_offset_in_orig_file> <8-seg_offset_in_backup_section> and then the actual backups (=copies of sections from the original file), one after the other. 2.3 XCD descriptor: - The XCD Header descriptor is a string in the form of "XCD Header v" where is a 'free' decimal number with decimal point. the string is case sensitive. - Current version is "XCD Header v1.0" 2.4 although the main aim for FLX is to use a mode 2 form 2 file, we know there could be circumstances (especially in CLX), in whice m2f2 file is not practical. i.e. when the whole file need 'backup' as in jpg or giff files. eventhough we don't have explicit header flag to indicate whether the media file is m2f2 (normal) or m2f1 (unprotectable), we decided on the convention that if <8-number_of_backup_segments> is 0 (zero), then it implies that the media file is mode 2 form 1 file (logic: -> ) 2.5 sample header file for an original file named "TheMatrix.avi", with size 780,000,000 bytes (~780M) offset |size | content -------|------|-------- 0 32 XCD Header v1.0\0 //xcd header descriptor 32 256 TheMatrix.avi\0 //original filename 288 8 780,000,000 as _int64 //original filesize 296 8 304 as _int64 //offset of the backup section, points to (*1). //(in this example it's the next field) (*1) //backup section starts here 304 8 2 as _int64 //2 segments backed up. //table entry for first backup segment 312 8 2048 as _int64 //this segment is 2048 bytes long 320 8 0 as _int64 //it backs up bytes starting at offset 0 of the original file 328 8 56 as _int64 //the backup in this file is at offset 56, //relative to the backup section (*1)=(@304) //table entry for second backup segment 336 8 65536 //this segment is 65536 bytes long. 344 8 770,000,000 as _int64 //the location in the original file. 352 8 2104 as _int64 //offset of the backup, relative to the backup section (2408 relative to 304 = 2104) //actual backups 360 2048 //actual backup for first segment 2408 65536 //actual backup for second segment 67944 0 EOF