ROMVault 2.0.0 - 模擬器

http://www.romvault.com/

Hi,
So around August time of 2011, I was working with the code of RomVault 1.7
and was starting to get frustrated with the direction it was heading. The
biggest problem was that the core data structured used internally by RV1.7
was based around trying to look like the list of DAT that it scanned in from
DATRoot. This sounds correct but I was running into problems like this: where
do I store the information about any files I find in a directory that is not
in one of the DAT directories. The scanning code was starting to do more and
more fixing, so that it could complete a scan, like copying unknown
directories and files over to ToSort. Also as I am sure all you RV1.7 users
know, there was no clean way in the data structure to store the status of the
Tree of DATs as displayed in the UI, so it kept on resetting itself, as it
was not really a tree structure at all, just a list of DATs displayed as a
tree. This was all heading in the wrong direction fast.

So I realized I had the change the core data structure, as what I am really
working with is actually not DATs, I am really processing a structure of
Files and Directories. (The DATs are just a way of representing the list of
files you want to collect.) So I started to rework the core data structure to
be a structure that would hold a tree of Files and Directories, and at that
time back in August 2011, I figured a couple of months of work, build the new
core structure, put all the old scanning and fixing code back on top of it
and RomVault 2.0 would be ready to go.

The new core data structure was made, and then I converted over the DAT
loader, and then got the File scanner going, and everything was going along
good and then I started working on the Finding Fixes code, and then got into
a world of CRC's, SHA1's and MD5's, it was probably around 10 months ago we
(that旧 me and my great Beta testers) realized that just fixing on CRC旧 and
file size alone was not going to cut it. So after a couple more month Levels
of scanning and fixing where born, and then after a few more months Levels of
scanning and fixing where about working, and then the improved DAT loader was
developed, and oh yes, the Missing report code was never converted over, and
then the idea of supporting filenames greater than 260 characters came along,
and fast Zip processing was added.....

You get the idea, the project just kept on snowballing. So here we are today
around 16 months later, with what is RomVault 2.0.0, with 40'ish private Beta
version being tested, about 300 check-ins to the source control. (So that
works out at about one check-in every 1.5 days for over 18 months!!!,You did
already click on that Donate button over there of the left, I hope.)

Is RomVault 2.0.0 finished? No, nothing is ever finished, there is a great
big list of things still to be added to it, but it is a great big step up
from 1.7. So before you ask there are 2 things still at the top of the to-do
list, so Sorry, there is still no 7z support, and still no CHD support. But I
will put this another way, neither of those 2 features would ever have fitted
into the 1.7 data structure, and so also look out for the 2.1 development
cycle starting. (As I actually already have a private test version of
RomVault that does to a limit degree support 7z.)

So you need now to know a little about scanning levels, and how RV works.
First you need to know what a CRC32 is, and to put this as simply as possible
it is a 4 byte, 8 digit (in hex) hash of your file, ZIP uses and stores the
CRC32 of your file when you put a file into a ZIP, this way when it extracts
your file it rechecks the CRC32 to see that your file did not become corrupt.

And here is an old DAT file game description, showing the crc values of the
files needed for this game:

game (
name 2020bb
description "2020 Super Baseball"
romof neogeo
rom ( name 030-p1.bin size 524288 crc d396c9cb )
rom ( name 000-lo.lo merge 000-lo.lo size 131072 crc 5a86cff2 )
rom ( name sp-s2.sp1 merge sp-s2.sp1 size 131072 crc 9036d879 )
rom ( name sfix.sfix merge sfix.sfix size 131072 crc c2ea0cfd )
rom ( name 030-s1.bin size 131072 crc 7015b8fc )
rom ( name 030-m1.bin size 131072 crc 4cf466ec )
rom ( name 030-v1.bin size 1048576 crc d4ca364e )
rom ( name 030-v2.bin size 1048576 crc 54994455 )
rom ( name 030-c1.bin size 1048576 crc 4f5e19bd )
rom ( name 030-c2.bin size 1048576 crc d6314bf0 )
rom ( name 030-c3.bin size 1048576 crc 47fddfee )
rom ( name 030-c4.bin size 1048576 crc 780d1c4e )
)

So this CRC32 numbers become a finger print of your file, and RV1.7 used the
CRC32 it found in the zip file header and file size to identify your files,
so if you have a file in ToSort RomVault would look at the header of the zip
file and read all the CRC旧 and Sizes from that zip file and if a DAT
requires a file that matches that CRC32 and size, then RomVault would move
that file from ToSort to the correct zip file as described by the dat.

This is all super-fast an easy as the CRC32 was right there waiting for me to
look at in the zip file information.

But here started the problem, the more files you start to compare this way
you realize that CRC32 is really not enough, which brings us right up to date
with the very latest TOSEC dat sets:

Commodore C64 - Games - [D64] (TOSEC-v2012-12-22_CM).dat

game (
name "Elvira - The Arcade Game (1991)(Flair Software)(M3)[cr MHI][t +2
MHI][a]"
description "Elvira - The Arcade Game (1991)(Flair Software)(M3)[cr MHI][t
+2 MHI][a]"
rom (
name "Elvira - The Arcade Game (1991)(Flair Software)(M3)[cr MHI][t +2
MHI][a].d64"
size 174848
crc 298caa9c
md5 71700ed3cf36b0724e0dd8e68fe32ed9
sha1 0dde6ccb17ed13c05f7f3a03ee6d636f5ba56545 )
)


game (
name "Jimbo (1995)(CP Verlag)[cr FLT][a]"
description "Jimbo (1995)(CP Verlag)[cr FLT][a]"
rom (
name "Jimbo (1995)(CP Verlag)[cr FLT][a].d64"
size 174848
crc 298caa9c
md5 92e4a98d8d9c2234c066124765a5192e
sha1 ba3eb11347d159951d94d7a34a3d08c40b6d009c )
)

Two files from the Commodore C64 set, and as you can see they both have CRC
of 298CAA9C!!!
Our system fails, CRC is not enough, and so bring on SHA1 & MD5 two other
hashing methods, and as you can see much bigger longer hash values with an
almost infinitely small chance now of the SHA1 checksum failing to be the
correct file finger print.

So now back to RomVault 2.0
Level 1 scanning: still just scans the CRC/Size right out of the headers of
the ZIP files, this is super fast, and if you just want to fix a mame set as
quick as you can and are about to join the next mame torrent which will
fixing anything that may be wrong anyway, then this is still probably enough
for you.

Level 3 scanning: this then gets the full set of CRC,Size,SHA1 & MD5 of every
file it scans which is unfortunately much much much slower that the quick
Level 1 scan, as is has to fully extract every file out of every zip file and
perform all of the checksums on every file. (RomVault 2.0.0 does do all of
this in memory, so it tries to be as fast as it can.) But does of course
guaranty all your files are truly correct.

And then the better Option:
Level 2 scanning: (USE THIS ONE) Level 2 scans every file just the same as
Level3, getting and checking the full set of CRC, SHA1 & MD5, and the first
time you run a Level 2 scan it will take a very long time, but it also stores
the file time stamp of your zip files, and so the second time you run a scan
it checks the file time stamp, and if the file has not changed then it does
not recheck it, so the second time around it actually goes super fast again,
only slowing down to read any new or changed files.

There is also Level 1,2 & 3 fixing, which sort of works the same as scanning.
Level 1 fixing, only fixes comparing Size and CRC.
Level 2 fixing, compares size,CRC,SHA1 & MD5, but will still copy the raw
uncompressed data from one zip to another, for speed if it can.
Level 3 fixing, compares size,CRC,SHA1 & MD5, but will uncompress and
recompress every file every time.

So with all that said, use LEVEL 2 Scanning, and LEVEL 2 Fixing, and your
future should be good.

That旧 enough I am done with the introduction, download it, have fun, and I
will probably try and make some new videos before to long.

And one last thing, did I mention about clicking on that Donate button
already.

--