bsnes News - The Future of bsnes (2010-10-19) - 模擬器

http://byuu.org/

Talking about the future requires elaborating on the past.


Up to the present

I initially started bsnes out of frustration with all other SNES emulators
showing virtually no interest in emulation accuracy. This affected me
directly as a ROM hacker, when I would discover that my ROM hacks would not
run on real hardware. I also felt it would be a good opportunity to create a
really powerful debugger, something that was sorely lacking.

It is important to note right off the bat: I never desired for bsnes to be
the most popular SNES emulator, I simply wanted people to move to better
emulators, and introduce some friendly competition to get all emulators to
improve, as simply submitting patches and asking them to improve was not
working at all. Simple accuracy improvements even when they caused no
performance penalties were rejected solely because they would break fan
translations that relied upon said inaccuracies to work. The emulators had
become entire platforms unto themselves. And so I set off to create my own
SNES emulator ...

As it turns out, the problem of accuracy was not linear as I once naively
believed, but exponential. That is to say, if emulating 90% of games requires
200MHz, then emulating 95% requires 400MHz, 97.5% requires 800MHz, etc. In
other words, there are rapidly diminishing gains.

This did not seem to be much of a problem, as even six years ago the kind of
hardware needed to run even what bsnes has become was very cheap and
accessible to just about everyone.

I was further inspired by software such as Nestopia. Here was an emulator
that came out of nowhere, and managed to become one of the most successful
NES emulators around. It gave me the impression that most people cared about
accuracy.

Niceties aside and truth be told, the most popular SNES emulators are to this
day still less accurate than NESticle was for the NES. The hacks are just
added inside the emulators instead of onto the ROM images directly these days.

Unfortunately, correlation is not causation. I failed to understand what it
was that made Nestopia popular. One has to look at the time period: NESticle
had very poor Windows support, very few people were able to get any audio out
of it at all with Windows XP and onward. People weren't switching for the
accuracy, they were switching because NESticle was no longer usable.

Speed could have also played a small part here. While NESticle could run on a
50MHz system, Nestopia needed 800MHz. But 800MHz was nothing even then, so it
was overlooked. Couple this with most peoples' inability to comprehend that
accurate emulation overhead is based around processor synchronization and not
raw clock speeds, and what emulation accuracy even means (it is essentially a
buzz-word these days), and you are put in a frustrating situation where
people wonder why dynarec/HLE N64 emulation requires less resources than an
accurate SNES emulator, yet they never question why an NES emulator also
requires more processing power than UltraHLE and its ilk.

Rather than being understanding and rational as I suspected, it turns out
that most people are instead highly aggressive and defensive when they
discover that their computers are not fast enough to run certain software
programs. Rather than simply admit that perhaps their hardware is showing its
age, they lash out at the software developers, completely oblivious to the
realities of what emulation means, and blind comparisons are made to other
software programs with completely opposing philosophies as justification as
to why said software programs should run faster.

But this still wasn't obvious to me. I kept trying, perhaps I could win them
over on features. I compromised on some of the very ideals I set out to
improve in the first place. I added support for countless compression
formats, even ones that nobody else supported. I even helped fix JMA to
support 64-bit systems, something the format author wouldn't even do. I added
support for the defective-by-design IPS patching format. I added all of the
filters, all of the fancy GUI features, save states, rewind, movie recording,
pixel shaders ... truth be told, at least half of my time developing bsnes
was probably sunk into improving the GUI and feature set.

But that was all in vain. I then came to the belief that perhaps it was just
speed. Now seriously going against my own philosophies, further becoming my
enemy to defeat him, I created alternate cores to push speed forward as much
as possible while still keeping accuracy above anyone else. This was a
blessing and a curse as it allowed me to also go back to my roots a bit and
work on my long discussed clock-based PPU video renderer. But I was now
essentially maintaining three emulators in one: the accurate version, the
previous version, and a new faster version.

And all along, I was supporting that trainwreck GUI code with so many
ridiculous features, and working around all of the bugs incurred by making
the terrible decision to go with Qt for the GUI toolkit.

And despite all of these sacrifices, and all of my efforts, nothing has
changed. Sure, maybe I attracted a few hundred extra end users, but they
didn't help report emulation bugs, it was still the same core group of people
who cared right from the start that helped me improve bsnes.

But at this point, the latest bsnes has 7,000 downloads, while the latest
ZSNES, the worst of the worst in terms of accuracy, and no better since I
started on bsnes back in 2004, has 26,000,000 downloads. By any measurement,
I have failed miserably to advance the SNES emulation scene at all.

ZSNES and Snes9X did not have the problems of compatibility that NESticle
had. Combined with peoples' familiarity, unwillingness to try something new,
and newfound sense of complacency when it comes to upgrading computer
equipment, there was never any chance for me to carve a new niche or improve
anything here.

I've come to the realization that there is absolutely nothing I can do to
convince people of the merits of accuracy and standardization to superior
file formats. Even if I were to get bsnes to be faster than ZSNES, and more
portable than Snes9X, it would make no difference. The name recognition and
familiarity is already there.


Motivation

So this leaves me with the realization that I have wasted years of my life
chasing something impossible.

Again, I want to reiterate that it wasn't my goal to become the most popular
emulator. I'll elaborate on exactly what I wanted.

First, as I said, accuracy. Why do I care about accuracy when most people do
not? Because I spent eight years prior to bsnes as a ROM hacker, tirelessly
hacking and translating games never released in English so that others could
play them. I successfully translated Dragon Quest V and Der Langrisser, and
failed at many more. But to me, it was really important that my work was
playable on real hardware. It went to reason that eventually emulators would
improve (and I still believe that eventually that will happen, even if it
requires the complete abandonment of the current popular emulators and the
architectures they are compatible with), and these translations would no
longer run.

I didn't just care about my own hacks. I bought a copier and got those
running on real hardware easily enough. No, I cared about everyone's hacks.
There were a half-dozen translations at that point which had problems running
on real hardware. It was the norm, in fact. The ones that worked fine on real
hardware were the exception to the norm. That, or very simplistic in nature.

I knew how much it'd bother me if my own work didn't run on real hardware,
and I didn't want other people to run into this problem where their hard work
would eventually be lost to the sands of time as older emulators were finally
deprecated.

The commercially released games themselves were also important, of course.
Most people may not notice that Starfox and Star Ocean battles ran at twice
the speed of real hardware, or that Earthworm Jim 2 was not playing any sound
effects. In fact, most people only care about the dozen or so games they
play. The rest need not even exist. So long as it's somewhat close to what
their fuzzy memories of decades past recalls, they are happy.

But I really cared about the games, and wanted these gems to be perfectly
preserved. The work of ROM hackers was no better than the work of the
original game programmers. They were just as entitled to perfect preservation
of their efforts.

I thought we had hope with Snes9X v1.52, but this was for naught. The focus
was placed on using a cycle-accurate DSP core, causing a significant speed
hit, but this was pointless as a cycle-accurate DSP core is useless without
cycle accurate CPU or SMP cores. Arguably the least important chip required
to be perfect was the first goal. The rationality was apparent: sound
emulation is often complained about, so do the least amount of work possible
to improve it. Yet it didn't work. Despite APU timing hacks for nearly 50
games, no amount of hacks can render Earthworm Jim 2 sound effects playable,
despite the fact that I was able to produce perfect sound in this game with a
sample-based (32x less accurate) DSP core from anomie, years earlier. At this
time, the Snes9X team has no interest in replacing the aging CPU or SMP cores
with more accurate ones.

Another goal of mine was to standardize and improve the way we stored
cartridges. Other emulators, which only cared about popularity, would simply
add support for every ROM format extension ever used. There are seventeen
different extensions still used out there in the wild because of this. One of
them is ".bin" ... it has made a mockery out of the ROM loading windows, with
constant false positive matches. The most popular extension is still ".smc",
which was the extension used by one of the most obscure SNES copier devices
made, the Super Magicom. Virtually no one uses ".sfc", which stands for Super
Famicom, the official Japanese name of the SNES. And since the Japanese
created the Super Famicom, it goes to reason what the extension should be.

When I asked the ZSNES developers if we could at least be somewhat reasonable
and remove the most ridiculous extensions, ".bin" included, I was told this
would not happen. But hey, at least they were slowly removing copier
interleave file formats. Not all of them, mind you. Just one more cause of
problems. But at least they were finally removing a few of them.

And then there were the ROM copier headers. These were 512-byte blobs of
copier-specific data that various copiers would use to load dumped games off
of floppy disks. They are completely useless to emulators. In fact they
aren't really even needed by copiers, the copier developers were just being
lazy when it came to loading the games.

The ridiculouslessness of this is aparrent when you realize that maybe 1% of
SNES gamers in the world even own a copier. And for the 1% that do, there are
dozens of copier header formats. The odds of downloading a game, and having
it have a copier header for your particular game are next to none. You pretty
much have to have a tool, like ucon64, to add your desired copier header and
split the games to fit on several floppy disks anyway. There was never any
reason for the raw cartridge data to store copier headers. All these copier
headers do is cause problems, the most obvious being that raw file checksums
and file sizes are meaningless since the copier header can contain any data
it wants, and may or may not even exist.

Combine these headers with ROM hackers and tool developers who could not
agree on whether to rely on copier headers, and Cowering who couldn't decide
whether his GoodSNES ROM verification tool should use them or not, and it's a
total crapshoot whether your ROM has a copier header, whether your IPS patch
requires one or not, or whether your ROM hacking tool requires one or not. Of
course, it only takes someone with a brain five minutes to realize there
should not be any copier headers, but that's asking too much I suppose.

When I suggested we remove these headers, I was bombarded by posts of people
thinking I was the new Hitler, by forcing them to remove their dearly beloved
and valuable copier headers, or else. No dissenters allowed. It boggles my
mind how people can be so willfully ignorant.

And then there was NSRT. Rather than help fight against copier headers, Nach
chose to co-opt them, storing NSRT data where copier headers appeared, to
give the emulators some special information about the games, like its opinion
on what the official names of games should be (not even Nintendo has a
consistent named list of games), and what types of controllers it supported.

I don't doubt that it is useful information, and that emulators probably
should be able to take advantage of such information to improve the user
experience, but storing this data in binary format in front of ROM data is a
stupid idea. Put it in a separate file.

If there was one thing that should have been stored inside the ROM file
itself, it would be ROM mapping data. You see, these ROM dumps have no
information on what memory mapping chips were actually used by the games
themselves. So as a result, emulators have created these elaborate hacks that
scan all kinds of arbitrary bits to create the memory map, and you get this
kind of hodge-podge map that doesn't really match what the real carts were
like. One more problem for ROM hackers who would eventually like to see their
games running on real cartridges.

Some would convince you that there is enough data to recreate these maps from
the internal headers required by Nintendo. Sure, but you're effectively
checksumming specific portions of the file to pick the right maps. Some will
say the data tells you exactly what is needed in a standardized format, but
there are two major problems with this: one, real hardware does not depend on
what is in the internal header ROM data, that can be anything. And frequently
is with those beta ROM dumps floating around like with "Batman vs The Joker".
And two, looking at the case of "Wanderers from Ys" versus "Fire Emblem:
Thracia 776", you cannot deny that the SRAM mapping trick is not part of any
standard. It's a blatant hack.

But even still, I knew that it was irresponsible to put this data into the
copier header. I felt that the best course of action would be an internal
database to recognize validly dumped games (which would also help prevent
false bug reports), along with external mapping definition files for ROM
hacks and translations. I even compromised to remove the idea of an internal
database entirely, and made the external mapping format fully flexible to
support any known mapping configuration. I spent weeks trying to discuss this
format, and the furthest it went with the ZSNES devs was nitpicking what we
should call each memory mapping mode ... who gives a fuck about nomenclature?
Pick something that is reasonably good, standardize on it, and move on.

Unfortunately it was a lost cause. Even after creating a standard and trying
to take into consideration the very small amount of input anyone would give
me, it went nowhere. No other emulator supports any form of external memory
map specifications.

Next there was patches. IPS patches are a miserable file format, and copier
headers only confound their problems. It's basically a coin toss with IPS:
you have a one in two chance that your patch will even apply to the right
areas in the file, because if the IPS patch was made on a headered ROM, your
ROM also needs to have a header and vice versa.

This was another trivial problem for other emulators to solve. Simply remove
the header prior to applying the patch. Problem solved, people would make
their patches without headers, and they'd work everywhere. But no, that'd be
too easy.

But even then, more problems were exposed: hard-patching was not reversible,
and there was still no way to tell if the patch even applied successfully, as
there was no checksum to compare against in IPS to tell. Given that many
games have multiple revisions, this further decreased your chances of getting
a patch to work.

I spent over a year working with Nach to fortify his NPS patching format. I
spent a year on the most popular ROM hacking forum discussing this new
format, UPS, asking everyone to please give input so that we could make it a
standard. Virtually no input. Finally, after over a year, I released the
finalized spec, and immediately got some ridiculous complaints about applying
multiple patches, something that is just as easy with UPS as it is with IPS,
and that was it. Virtually nobody distributes UPS patches in the SNES scene.
It's only really caught on in the GBA scene where ROMs larger than IPS can
support are common.

It is a source of great frustration that the person who initially came up
with the file format has so far done absolutely nothing with it. No tools, no
support in his emulator for it, nothing.

There is UPS support in NES, Game Boy, Game Boy Advance and Genesis
emulators, but no support in SNES emulators, where patches are the most
common. It only exists in Snes9X v1.52 because I finally broke down and added
the support myself; and zones, who was never made an official mainline
developer, was nice enough to add the patch once the rest of the Snes9X team
abandoned the project, leaving him holding the bag.

And then there were cheat codes. Here is a format that limits you to very
short, ANSI-only strings for descriptions, and has no support for grouping
cheat codes. Ever see those "3CA7-1BF2+2A1D-98F3" multi-part codes? Sure
would be nice to group them so you only have to toggle one to turn a code on
and off. ZSNES is even worse off, as the emulator only supports a 5x5
all-caps font due to its legacy DOS design, you can't even use lowercase in
your descriptions. Further, these files were binary-based, so you couldn't
even edit them yourself outside of the emulator.

I tried to work with the other emulator authors to come up with a sane format
that eliminated all of the old problems, again to no avail. I had to push on
and create an XML format that supported UTF-8 text and unlimited-length
descriptions on my own.

And it just goes on and on. All I wanted was to make everything better, more
enjoyable for everyone. But I was fought every step of the way by just about
everyone, absolutely and completely resistant to any kind of change from the
status quo.

Wasted effort and the straw that broke the camel's back
So back to my original point about ensuring games work in the future on all
emulators. Even after five years of bsnes being out there, and me preacing
these virtues everywhere I could, we had the Lennus II translation in 2009. A
fan translation that only ran in ZSNES.

I really tried, I put my vitriol aside, stepped in, and offered to help. I
offered to work on the game, without even having the ROM hack source code, to
track down and fix the problem. Only to come across a post by one of the
authors on another forum some weeks later essentially calling me an asshole
for being pedantic about it not running anywhere else, all the while putting
"accuracy" in quotes as if it was a made-up term. Talk about gratitude. When
I responded, it was of course denied that he was referring to me, the only
person who ever brought it up. But I wasn't born yesterday.

Still I pressed on. Until recently. I came across yet another Chrono Trigger
hack. This one had it all. Apparently the developers wanted to add CD-quality
music. Hey, what about MSU1? That's a great, simple way to add support which
can then be added to other emulators trivially. Nah, they instead decided to
create a ZSNES fork that monitored RAM variables to play music externally.

Not only does this permanently lock the hack to ZSNES alone in a way that you
can't possibly add this support to any other emulator, it also makes it
impossible to maintain an accurate pitch when interacting with the emulator.
Fast forward, rewind, pause, entering menus, etc ... forget about it.

Okay, well at least the patch will be UPS right? Nope, IPS. And it requires a
ROM copier header, too.

Well, at least by this point people will be upset that it doesn't work
anywhere else but ZSNES? Nah, maybe one person, who was subsequently ignored.
"Who cares? ZSNES is perfect, use it" (even though you may not be running on
an x86 32-bit compatible operating system, of course.)

And you know what? Despite me being one of the most vocal people out there, I
didn't even care. I didn't even have the slightest ounce of concern to bother
responding. I had had enough. I've done everything I could do by this point.
All for nothing. These people are fucking retarded. You don't care? Then you
know what, neither do I. When nobody can run your game in ten years, that's
your problem, not mine. You can't say I didn't do everything in my power. But
some people just can't be helped.


Toward the future

That led me to my next epiphany. I've been wasting years of my life chasing
some foolish dream, all for nothing. The people who could ever care already
do, the rest never will. All I've done was get upset and depressed at the
constant barrages about how bad my emulator is because it won't run on their
eMachines PC from 1999, and what a jerk I am for posting about why accuracy
matters. All I've done was compromise my own values by trying to build
bridges and meet at the middle ground. All in vain.

Why? Why bother anymore? I don't have it in me to care anymore. I've given
up. bsnes is no different from any of my other projects: doomed to obscurity.
But you know what drives me to spend seven years on Der Langrisser, only to
have about 1,000 people download it? What drives me to work on bsnes for six
years only to have about 7,000 people download it? It's because I do it for
myself, and not for anyone else. I personally enjoy it.

And that is why I'll never give up, no matter what is said, and no matter how
much people try to hurt me. Despite the fact that bsnes has always been 100%
free software, and given away without asking for anything at all in return.
Peoples' sense of entitlement truly frightens me.

So fuck it all. Enough with the compromises. The bsnes/phoenix port that
removes Qt is a new beginning. I have not added compressed archive support,
rewind support, movie support, IPS patching support, software filter support,
alternate ROM extension support or even copier header support. And I'm not
going to. I created the GUI I personally wanted in two weeks, not three years.

But even with this, looking at bsnes, I have to be honest. It's pretty much
finished now. Every single processor, even the PPU and DSP, are emulated at
the clock level. Even the add-on processors, the SuperFX and SA-1 ... all of
the caches are emulated, all of it is emulated right down to the individual
clocks. With 100% compatibility and no known bugs, it's about as perfect as
it's ever going to get. The things I am improving now will never even be
noticed by anyone.

That's not to say it is 100% perfect. New bugs continue to pop up now and
again, and I'm very grateful to anyone who will use bsnes to spot them,
report them, and help get them fixed. I'll continue working on bsnes to keep
fixing any discovered issues, and to keep improving even those things that
nobody notices, for as long as I possibly can.

But I'm done catering to others. I'm done trying to convince the world of the
merits of accuracy and superior + unified file formats. It's like talking to
a wall, and I don't need the pain and disappointment when people go on the
defensive attacking me for it.

Going forward, I may or may not release Qt and/or phoenix-based Windows
binaries. Who knows. I'm not going to destroy the Qt port, it would be stupid
to throw away something already made. I'm just no longer going to keep
improving it to add requested features.

Instead, I'm just going to keep focusing on the emulation core. The source
code is free software and open source. People can do whatever they want with
it: continue the Qt port on their own, distribute their own compiled
binaries, create their own GUIs, add all the features they want, and that's
cool with me. Or they can create their own emulators to compete. I don't care
anymore.

I will say one thing to any potential future competition that indicates that
their emulator is more accurate: they have either stolen my core, or they are
full of shit. I've spent six years performing countless tests, and had the
help of a half-dozen geniuses during that time like anomie, blargg and Jonas
Quinn. There is nothing short of one-off contrived examples that would pass
on another emulator but not on bsnes. And nothing above what I've done
already will ever make any visible difference in any commercial games.
Perfection may very well be impossible, but I'm as close now as one can ever
realistically get. I've taken accuracy from probably 70-80% all the way up to
99.99%. Debating one-thousandth of one percent of a difference is pretty
asinine, even for me. So there is really no point in engaging in competition
anymore. Still, I do hope more people will step up and offer emulators that
can equal bsnes. I do still want the scene as a whole to improve, I'm just
done with trying to do so myself.


In closing

I formally withdraw from any and all efforts to promote bsnes and/or its
ideals in the public space. From this point on, the only forum you will see
me participating on is my own, board.byuu.org. bsnes the public project is
dead. Insult me and my project all you want, I'm not bothering anymore. Do so
elsewhere and I won't even see it, do so on my forum and get banned.

bsnes is now my personal project, and I am only uploading the source code for
those who do care about the things I care about. The rest of you who would
complain about my efforts, can go straight to hell. I owe the world nothing
at all.

If you want my help for something SNES-related, or if you want to help
improve bsnes by testing it and reporting bugs, please don't hesitate to stop
by my forums.

And lastly, I do want to apologize for all of my anger of the years. It's
been a direct result of the things I've discussed above. It's because I cared
so much that I let it get so personal, and get me so riled up in the first
place. And that is why I am stepping down from the public space.

--