Disk full with RRG + MJPEG Codec

[tiqq]

Recording files greater than 4 GB causes a disk full error !!
I know this is a 32 Bit limitation - but who is responsible for it ?? The application or the codec ??
The WIN32 API supports 64 Bit file IO !!!
Why it´s not supported ??

I could easily record files about 25 GB.

I am using the VideoTools 1.5 Beta with the RRG & G400 Max.

Any help ??

tiqq

[Grigory]

Avi file has a field in the header that contains 4 byte number with the meaning of a distance from the beginning to the end of avi data in bytes.
When a program like Premiere writes in avi, it does it job continuously adding avi data to the end of file. When the job is done, it tries to write that digit in the header.
If the digit is invalid (has >2G value, interpreted as negative), an error occurs, and Premiere says that the disk is full.

If you interrupt the program by force, using three key combination and END TASK command, the file data may be good, but a header length number is not written at all. The file is reported as corrupted.

If the video capture program like Premiere or VidCap writes into pre-allocated avi file, the actual data size inside avi may be less than pre-allocated file space. You get very large file with correct header information. Here you can get another funny result: file properties report fantastic data rate of many MB/sec, obtained by division of file size by reported in the header file duration. Premiere "get properties" command, however, provides correct info about the data rate.

Asf format can use 8 byte length specifier, but there are no tools to edit in asf. BTW, you can easily convert avi to asf without changing the codec. MS did not gave us a tool to produce one asf from several avi's. If it was possible, we could make unlimited length MJPEG or DV (or any other codec) asf files, which are perfectly playable on PC.

It may be possible to recover <2G portion of corrupted AVI by carefully writing its data size in appropriate place. However, I do not know where and what to write.
I remember that some programs can recover part of avi written for example on a ZIP disk with media defect (read error). I could determine the duration before defect and after media defect and write two avi files with only few frames lost.

Many of us sometimes get trouble while rendering ~2G files and getting error near the end of rendering, because the output file happens to be 10 kb longer than 2G. It will be very nice if somebody can write a small utility that changes the header to a valid <2G value. In this case we can escape re-rendering of all file from the very beginning. I hope it is possible.

Grigory

[This_Idiot]

Hi,

Nice discussion. Can I just add a couple of things. First, although Win95 does support FAT32, this is not the native disk format, and has to be deliberately invoked.

Secondly, Grigory's suggestion about an "adjuster" routine is very sensible. One would need to know exactly what byte offset/length is used for the header info, and then some kind of routine to read/rewrite the header (possibly also need to truncate the end of the file to the required size). I'm sure that Markus would be able to give us details of the header info, so all we need now is a programmer with the right kind of software to do the biznis. I don't think I have time to hijack a mainframe Cobol compiler from my workplace before I leave tomorrow, and porting it to a PC would work If there are any C/++/VB/Assembler gurus out there that fancy their chances, we would all be grateful for input.

[Grigory]

The AVI file cannot be >2Gb in size. The header inside the file cannot describe more than 2G bytes. If you write longer file, the header length information contains incorrect number.
It cause unexpected system responce, like "disk full". You can get many more variants, but the rule is:

ALWAYS make files < 2G in size. This is MS limitation for AVI.

Grigory

[tiqq]

It´s a file spec limitation ???
Oh boy... and that´s the only one to use !!

But, by the way, premiere reportet after 4 GB the disk full error message - not after 2 GB !!

By the way ...
now I know why the remote control produces files < 2 GB.

Thanks,
but I hope there will be a better file fomat - soon.

tiqq

[Spire]

Let me elaborate on Grigory's post.

AVI file format limitations:

1. The AVI file format uses 32-bit values internally, limiting the file size to 4GB.

2. Many (most, actually) AVI parsers erroneously use signed 32-bit variables to work with the 32-bit values, chopping the file size limit in half, to 2GB.

3. A few other AVI parsers are further "broken", and choke on anything larger than 1GB.

File system limitations:

1. The FAT16 file system (supported by DOS 2.x and up) absolutely cannot handle files larger than 2GB.

2. The FAT32 file system (supported by Windows 95 OSR2 and up) absolutely cannot handle files larger than 4GB.

3. The NTFS file system (supported by Windows NT) can support files of any size.

The bottom line: As Grigory says, do not try to attempt to work with AVI files larger than 2GB. If you need to capture large continuous amounts of video, use a program such as <A HREF="http://www.nct.ch/multimedia/avi_io/">AVI_IO</A> that can capture into a series of AVI files, "spilling over" the data into successive files when necessary.

[stealth]

Yes, nice discussion. During the summer holidays I needed to obtain the AVI file format, I captured something from TV, and the resulting file was longer than 2 GB, but shorter than 4 GB, however it was unusable in any program (player, editor, encoder). Analyzing the AVI format I found 3 problems with it.
1) AVI header is formed as "RIFF"<size of RIFF block: DWORD>"AVI "
2) movie section header - after AVI header there is movie section with video/audio data
and header "LIST"<movie section size: DWORD>"movi"
3) usually at the end of the AVI file, there is an index (for faster access), any item of index looks like <block_id: FourCC><some_flags: DWORD><block adress: DWORD><block length: DWORD> and the block adress is the problem.

After analysing the AVI format I wrote a litle program for splitting AVI files (it is tested only with MJPEG codec). It can repair some damaged AVI files, which were created by the capture program using signed integers.

I can send/upload this program somewhere if you want.

Good luck,
StE[a]LtH


------------------

[stealth]

Whoops! I used the 'less' and 'greater' signs around the important expressions, here is the fix:
1) avi header "RIFF"riff block size: DWORD"AVI "
2) movie section header "LIST"movi section size: DWORD"movi"
3) index item - block_id: FourCC, flags: DWORD, block_address: DWORD, block_length: DWORD

StE[a]LtH

[Grigory]

Excellent!
But too complex for my old brains...

What to write and where?

[This_Idiot]

Stealth,

Email me the proggie and I'll put it in the downloads section. If you can also write up a noddy user-guide, it will save an awful lot of queries.

Thanks

Chris

[BrownEyes]

Quite some time ago when I was capturing Startrek Voyager series, I had the same 2gb limit problem. The specific resolution and compression I used could only give me 'bout 30 minutes of video (one voyager segment= 44 minutes or so). Then I converted the two avi's to mpeg. So far so good. But what I realy wanted was one mpeg file, so far I haven't found a way to merge two mpeg files...Does anyone know how to do this?

Ramon

[Kevin Culp]

Ramon,

To ***smoothly*** join two MPEG-1 files, you really need to do it during encoding. In other words, feed an encoder the original AVIs and let it encode the MPEG as a solid stream. Unfortunately, the stand-alone versions of LSX-MPEG and Xing don't allow you to do this. You really have the following options:

- Load the files into Adobe Premiere and export to MPEG-1 with the Xing Plug-In. Of course if you don't have access to Premiere or Xing, this can be a very expensive option!

- Purchase Darim's DVMPEG, a true codec that allows you to export to a single MPEG-1 (or MPEG-2) file from the timeline in MediaStudio Pro, Avid Cinema, Premiere, etc. However, quality at lower bit rates is rather poor. DVMPEG isn't cheap, either.

- Use AVI2MPG1, a stand-alone freeware encoder. Quality isn't as good as LSX or Xing at lower bit rates, however.

If you can live with a brief "glitch" at the joining point, you ***can*** join your final MPEG files with IFilmEdit. If you split the files during a commercial break this shouldn't be noticable at all. You may also use Camel's MPEGJoin program or a DOS "Copy /b" command, but unfortunately both of these won't report the correct play time for your file and if you plan to put your MPEGs on VideoCD, some authoring programs will reject them.

Does this help?

Enjoy,
Kevin

[Grigory]

I remember the shareware program that was able to join MPEGS correctly. The demo version was writing the output into "somename.dll" file, which "could not" be saved by a button click. However I could rename this file or copy it to another location and get good MPEG .

Unfortunately, I don't remember the name of the program. And, I am not sure the author did not change the "protection".
I found this program in winfiles.com.

Grigory

[Grigory]

Another option: http://fly.to/dvd99 http://members.xoom.com/_XOOM/dvd99d...mpegrepair.zip