XCD Specifications: initial draft (29-Apr-2002) ----------------------------------------------- discussed by de_xt, koepi and avih actual document by avih General: -------- The XCD is a CD container format standard for multimedia content (much like VCD or SVCD). In contrast to S/VCD though, it allows arbitrary multimedia content TYPES (VCD/SVCD supports only MPEG1/MPEG2 system streams respectively and images). XCD can support, for instance, MPEG4, VP3, DIVX, XVID, JPG, etc.. ANY media type. It uses reduced Error Correction facilities, such that more space of the CD can be used for actual media (like S/VCD, compared with current computer CDs). Due to the reduced error correction, it's aimed at stream/media types that can handle errors to some extent at least. Yet, it allows keeping arbitrary 'special' sections of the stream/media 'protected' (i.e. special headers/indexes etc). It contains recommendations for the player as for the order of the streams to play (in the form of optional playlists). Targets: -------- - Allowing arbitrary stream/media containers/types (i.e. ogg, mcf, avi, mp3, wav, jpeg, gif etc). - To be playable by both stand alone players and computer players. - Keep the minimal player implementation requirements very low for the XCD overhead. (Naturally, it will also have to include actual decoders for supported stream/media containers/types). - Offering optional advanced features such as playlists, Interactive navigation (similar to CD-i) and more for players that support such features, and arbitrary binary information (files) to be used by computer users. Annotations: ------------ XCD = Extended Compact Disc EDC = Error Detection Code (i.e. CRC - allows to check if errors exist, but not to recover them) ECC = Error Correction Code (i.e. Some redundency to allow reconstruction of data which is [to some extent] damaged) mode1 = CD format: ISO filesystem for binary content (as used by todays computer CDs). Error Handling: EDC and ECC. m2f1 = CD format: Mode 2 Form 1 iso 9660 standard. Error handling: EDC and ECC. m2f2 = CD format: Mode 2 Form 2 iso 9660 standard. Error handling: EDC. Bridge= Special CD mode that allows both m2f1 and m2f2 files to exist on one CD (like S/VCD) TBD = To Be Defined = Stream Serial Number. Four numeric characters in ASCII format. Padded with leading '0' if necessary. = Playlist Serial Number. Same format as . 1. CD Layout 1.1 the CD is of Bridge CD-ROM XA Format, with both form 1 and form 2 files. 1.2 All the files comply with 8.3 naming convention. All caps. (Not including [BIN] content??) 1.3 The directory structure is 'flat'. i.e. no nested directories (Except for [BIN]???). 1.4 Each stream/media is to be identified by 4 numeric characters (Stream Serial Number aka ) Thus allowing up to 10,000 streams/medias on a single XCD CD. ( is in decimal format). The should be sequential (with no 'gaps') starting with 0000 and possibly up to 9999. 1.5 Directory structure: [XCD] - The bridge(s) to the file(s) saved as m2f2 track(s) ????(TBD) [HEADERS] - Extra streams/Medias info (as the original file name, media tags etc) and backup of the vital streams content (i.e. headers/indexes/etc) Naming convention: HEAD.XCH i.e. 'HEAD0007.XCH' [MEDIA] - Actual streams/media files as m2f2 files. Naming convention: STRM.XCS i.e. 'STRM0007.XCS' [PLAYLIST]- Predefined playlists for the content of the CD. Naming convention: PLAY.XCL i.e. 'PLAY0002.XCL' [BIN] - Arbitrary binary content for computer users. Could be used for Players, filters etc. [XCDI] - Place for interactive stuff like on CDI (TBD) 2. Files content 2.1 [XCD] - TBD 2.2 [HEADERS] - each HEAD.XCH file referes to a specific single STRM.XCS file. each XCH file is in m2f1 and has the following structure: char xcdDescriptor[64??] // TBD- i.e. "XCD HEADER V1.0" byte protectionScheme; // 0=default protection scheme, 1=predefined scheme, 2=manual scheme. char originalFileName[256]; // NULL terminated ASCII string. UINT64 streamSize; // BIG/SMALL ENDIAN?? (what's the x86 format??) TAG mediaTag[2048??]; // TBD- ID3, or other media tag convention. UINT64 numBackedupSectors // number of backed up sector that this header contains. {XCD_BAK}+ // actual backups for sectors of STRM.XCS each XCD_BAK section has the following structure: typedef struct cxd_bak { UINT64 sectorNumber; // Referes to the sector beeing backed up from STRM.XLS // the 1st sector of STRM.XLS would be 0 for that matter. UINT64 dataOffset; // Data offset in bytes, from the begining of the original file. // Applied AFTER stripping the headers and edc (or others) associated with this media. // This is redundant information to make life easier for the parser. char userData[2324] // Actual backup of the user data section of the backed up sector. } XCD_BAK Example of such complete header, named 'HEAD0008.XCH': offset(dec)| content -----------|------------------ | "XCD HEADER V1.0" // i added the descriptor and protection scheme after i finished the table... :( // so the absolute offsets are incorrect | 1 // predefined approved protection scheme. 0 | TheMatrix.ogg\0 ... // padded with trailing useless chars 256 | 780,000,000 // original file size as UINT64 (ENDIAN??) 320 | "blah blah blah" // tag that give auxiliary info about the stream. TBD 2368 | 2 // 2 as UINT64 (num of backed up sectors) 2432 | 0 // back up for sector #0: sector number as UINT64 2496 | 0 // (redundant) data offset of original data to be backed up 2560 | // actual backup: block of data, sized 2324 bytes 4884 | 1 // backup for sector #1. 4948 | 2324 // (redundant) data offset of original data 5012 | // actual backup for sector #1 7336 | the above header relates to the media stream 'STRM0008.XCS'. it contains: - The original name of the file ('TheMatrix.ogg') - The original size of the file (~780M) - Some extra information regarding this clip. - Backups for 2 sectors: sector#0 and sector#1 of the 'STRM0008.XCS' m2f2 file. 2.3 [MEDIA] - each STRM.XCS is a standard m2f2 file generated from the original media file, padded with trailing '0' if the size is not a multiple of 2324 (to match the standard sector size) 2.4 [PLAYLIST] - each PLAY.XCL is a m2f1 file, and contain a list of streams to play one after another. it may refenence other playlists, or notify to change the media (insert the next CD for example). it may offer a specific timestamp to jump to in the specified stream. it may indicate a delay to issue AFTER the item has been played. it MUST NOT be recursive with relation to other playlists on the CD. The structure of XCL file is as follows: the 1st line will be an xcd identifier i.e. "XCD PLAYLIST V1.0" typedef struct play_list_element { char itemType; // can be: 'S' for a stream on this cd, 'P' for another playlist or 'M' for a media change. PLAY_ITEM item; } PLAY_LIST_ELEMENT; typedef struct stream_pointer { UINT32 ssn; // UINT32 timestamp // in msec, i.e. 1000 for 1 second. UINT32 delayAfter // in msec } STREAM_POINTER; typedef struct playlist_pointer { UINT32 psn; // UINT32 delayAfter // in msec } PLAYLIST_POINTER; typedef struct media_pointer { UINT32 mediaNumber; // i.e. 3. will show a message "Please insert CD #3" } MEDIA_POINTER; typedef union play_item { STREAM_POINTER stream; PLAYLIST_POINTER playList; MEDIA_POINTER media; } PLAY_ITEM; *** another way to implement this can be as an ascii file, instead of binary, so it can be edited using standard ascii editor. feedbacks regarding you preference is welcome. 2.5 [BIN] - this directory contains arbitraty binary files, to be used by computer users. i.e. players, filters etc. 2.6 [XCDI] - TBD- this directory contains interactive stuff. like menus. may support 'overlay' of the menu on top of a specific stream or playlist. if you know of a free good menu system, let me know. otherwise, we'll invent one :) 3. Minimal Requirements 3.1 Mastering package minimal requirements: - create the correct cd format (bridg xa) and the [XCD] directory content. - create the correct directory structure (all directories are compulsury, even if empty) - be able to utilize protection schemes (regular expressions?) for different file types. - create at least one STRM.XCS file with matching HEAD.XCH file, acording sections 2.2 and 2.3 - minimal protection scheme (REGARDLESS of downloadable/manual/etc): the 1st sector MUST be protected. 3.2 Player minimal requirements: - be able to read the cd content (xa m2f1 and m2f2) - be able to associate header file with a matching stream file, and utilize the protections implemented in the header (such they are regarded together as a single COMPOUND_STREAM) - be able to identify (either by content or file extension) if it can parse the COMPOUND_STREAM - it's most recommended that it has at least decoders for a single media stream type. - fail 'gracefully' if the content cannot be rendered (i.e. if an ogg stream contain xvid video and ac3 audio, but the player doesn't have ac3 decoder, it should at least play the video, (assumming it has xvid decoder of course)) 4. Opional features: 4.1 Mastering package optional features: - creation of correct playlists - adding binary files to the [BIN] directory - create interactive menus. 4.2 Player optional features: - read and 'render' playlists correctly - render menus (either with or without 'content overlay') 4.3 note: To support the XCD format, the minimal requirements of the player is to be able to read and parse the [HEADERS] and [STREAMS] dirs. (Note however, that not all players will support all formats. i.e. some players may not support avi clips, while supporting mcf and ogg clips. portable stand alone players may support only mp3 and ogg streams if they wish, or support avi files, but render the audio part only, etc.) 5. protection schemes for specific stream types. we should aim at a repository of protection schemes (regular expressions), that allaow each file type to have different protected sectors. the default and minimal protection scheme is to protect the 1st sector of each stream. that might be enough for many stream types. ------------- end of initial spec -------------- ------------------------------------------------------------------------------------------ notes from koepi (before the above initial spec was written and after a short discussion): First draft of the XCD-specs from 29/04/2002 ---------------------------------------------------------------- 1. CD Layout: [HEADERS] - copy of the vital headers of the stream saved on CD [XCD] - the bridge(s) to the file(s) saved as mode2f2 track(s) [PROGRAMS] - Players, filters or any content like this [] [] [XDI] - Place for interactive stuff like on CDI ---------------------------------------------------------------- 2.1. Headers Depending on the file-type stored in the M2F2 track, the vital headers are stored in the mode1 area in a file called "secureXX.dat". The XX is placeholder for the possible multiple streams, starting with "00" for the first media content file saved as mode2 form2 track. E.g. for avi this file would include the first header, an index and the last header, for the OGG format this will contain the first 3 headers of the stream. We should save some extra information on the CD as well, stating the version of the spec (e.g. "XCD 1.0"), the real filename(s) (e.g. "The Matrix.avi" or "The Matrix.mcf"). 2.2. XCD Here we list up our media content files with their 8.3 "representatives" like streamXX.dat, where XX starts at "00" again and corresponds to the headers backup. -------------------------------------------------------- additional thoughts 300402 - avih additional [XTRAHEAD] and [XTRASTRM] for extra content besides the 'standard experience' content. i.e. clips for menu overlays, interviews etc. player support: XCD Level 0 - 1. identify the CD as an XCD, and able to read the directory structure. 2. able to read the header files of the standard and extra streams (for original name and filesize). 3. able to enumerate the streams (??) XCD Level 1 - 1. the player has built in decoder and renderer for at least one stream/media type. 2. enable the user to play each stream individually, either standard or exta. 3. enable the user to seek th stream, if the stream type allows it. XCD Level 2 - 1. able to play all of the 'standard' streams sequencially in one go (=standard experience) (only for streams that the player has decoder/s for) 2. full support for XCD playlists. XCD Level 3 - 1. support for XCD interactive menus. (supporting overlayed menus is optional, but the menu files must be parsed correctly anyway) Example: The new HellPlayer042 is fully XCD Level 1 compliant. It supports MP3 and Ogg streams. HellPlayer042 supports MP3 streams with or without CRC. HellPlayer042 supports Ogg streams with video of types DIVX4.xx/XVID/VP3, and/or Audio of types MP3/vorbis.