KEEP, a universal emulator or a complete waste of time?

In Depth - A merciless lens pointed on the hot topics, passionate and detailed retrospectives, reflections beyond the appearances Alerted by the inexorable advancing of the next digital dark age, the European academy started a preservation plan for digital contents and artefacts with great ambitions. The KEEP (Keeping Emulation Environments Portable) project considers with particular care videogames and intends to create what has been defined the first “general purpose” emulator, capable of providing access to obsolete media and formats for nowadays and future generations.

The need to archive the bit heritage of a mankind accustomed to digital language is an instance that is spreading among researchers, just look at the case of the National Videogame Archive hosted in Bradford Media Museum, middle England. KEEP, that involves several universities and researchers from Europe, follows however a rather different path because it embraces the principle of software emulation for obsolete hardware, to make the contents developed for that hardware be always available for anyone even when the original machines will have ceased to work.

Funded with 4.02 millions euros, the project describes the so-called “first” general purpose emulator like “a piece of software which can recognise and ‘play’ or open all previous types of computer files from 1970s Space Invaders games to three-inch floppy discs“. “Early hardware, like games consoles and computers“, says KEEP lead Dr. David Anderson from Portsmouth University, “are already found in museums. But if you can’t show visitors what they did, by playing the software on them, it would be much the same as putting musical instruments on display but throwing away all the music. For future generations it would be a cultural catastrophe“.

KEEP is therefore primarily designed to guarantee accessibility for digital contents, be they management software, arcade videogames, audio tracks or database files created with Lotus 1-2-3 for MS-DOS, the first “killer application” for the original IBM PC. “A vast bank of information needs to be catalogued and stored“, says the computer games expert and Anderson partner at Portsmouth Dan Pinchbeck, and among this data videogames keep a particular role because they are seen by the industry as “disposable, pulp cultural artefacts“, while instead they represent “a really important part of our recent cultural history“, “one of the biggest media formats on the planet” hence they deserve to be preserved “for future generations“.

Syndicate - PC DOS

Against the hastened aging of digital culture KEEP will ensure that the universal emulator will be constantly updated, to continue to perform its duty as a virtual archivist in the future. Among the components of the preservation system there will be the open source software Dioscuri, an x86 emulator written in Java managed by Nationaal Archief in Netherlands, and specifically designed to be extensible and agnostic to any platform, guaranteeing in this way that the emulation will continue to work in the future without being trapped in a single computing architecture.

If someone had a déjà vu at this point, such impression couldn’t be more correct: Dioscuri isn’t anything but a stripped-down version, as for performance, capability and documentation of MAME, the arcade machines emulator which popularized classic videogames preservation instances since 1996, way before the members of the Pan-European academy would start to just speculate about the need to avoid the forthcoming digital dark age.

Insofar as MAME is an intentionally specialized project and not a “general purpose” one like Dioscuri/KEEP, it collects the essence of a knowledge polished up during time, often obtained with “epic” reverse engineering actions against absurd protections from the producers (just look at Capcom and the case of the infamous CPS-2), concerning the microcode of hundreds of different CPUs, logic circuits, PALs, input ports, control and memory management interfaces and much more.

Such completeness is also the ambition of MESS, MAME’s twin project that starting from its code base tries to emulate anything that haven’t gone through the arcades, be it a computer or a videogaming console. MESS is much more closer than MAME to the universalist model of the KEEP project, and in this case too the emulator code contains an amount of information that represents, without any doubt, the state of the art as for accuracy to replicated hardware, robustness and portability on the several computing platforms available nowadays.

Battle Chess - Amiga

Both MESS and MAME are in fact designed with the specific aim of running on anything have enough power, the PSP portable console, PlayStation 2/3, Xbox 360, netbooks, Linux, Unix, Amiga, OS/2, x86, RISC, MIPS systems and even a decrepit oscilloscope. The same thing can be said about DOSBox, the best emulator of Intel machines capable of simulating an 80486 CPU, a CGA/EGA/VGA graphic card, a MIDI/Sound Blaster sound card and lastly an MS-DOS compatible shell. ScummVM isn’t exactly an emulator, but it’s dedicated to portability and preservation of classics from the past as much as the aforementioned MAME, MESS and DOSBox are.

The essence of the matter is that floating around there is the availability of so much earlier knowledge about emulation, be it specialist or universalist it doesn’t mind because in a given period of time the elements used by producers (CPU, sound DSP and so on) tend more or less to be the same in the arcades, on computers and home consoles. A this point one could expect that the KEEP folks would enthusiastically take from such knowledge for their monumental archiving effort, on the contrary by reading discussions on the forums it just seems that the European academicians have no intention to dirty their hands with the hobbyists’ work and want to do anything by themselves.

On VOGONS (DOSBox official forum), MAMEWorld, on the MESS message board it’s highlighted the fact that, even if the researchers have been in contact with many emulator developers (MAME and MESS ones in particular), they have no interest for code and “dumps” of hard disks, floppy disks, ROM memory chips, BIOS and all the rest of so many digital contents already converted in binary files to run within the aforementioned emulators, hence they want to do everything again from scratch, from the emulation code of the Space Invaders microprocessor to the enormous study and re-engineering work already made by the vast community devoted to videogaming preservation since two decades.

In brief the KEEP folks would like to reinvent the wheel, as MAMEWorld’s admin Smitdogg states, basing moreover their efforts on a coding language, Java, incredibly inadequate for intensive and complex applications like the emulation of any full console or PC. Mamedev Smf, who dealt with code and emulation pretty well in these years, explicitly talks about burnt money and scarce hopes to succeed considering the approach chosen by the European researchers.

Super Ghouls'n Ghosts - Super Nintendo

KEEP resembles much more an aberration born to advertise the cloud computing hoax than a project destined to achieve its goals within the estimated 36 months (from now to 2012). Theoretically it’s a wonderful endorsement initiative for commitment and passion of those following emulation since years and the videogaming one in particular, in practice it appears to be an on-line demo technically very limited and destined to end soon unremembered.

10 comments

  • Java, and duplicating effort. That sums up this project really well.

    It’s such a facepalm to see big money trying to match what dedicated hobbyists have already accomplished.

  • Heh, one can only imagine what could have been achieved with those millions of euros by people that worked out DOSBox, MAME, MESS, ZSNES, PCSX and the other projects often by betting their money on them…

    Maybe the secret is always the same: put passion in something and you will succeed at (almost) any cost, put money in it and you will get what you have paid for. CPS-2, PS2 and the other countless reverse engineering efforts are worth way more than 4m euros, from this standpoint…

  • Jorpho

    DOSBox might be an excellent program for running old DOS games, but I really don’t think anyone ever said anything about it being the “best emulator of Intel machines”. Not by a long shot.

  • Fritz B. M.

    I find dosbox emu very useful. And its a lot more than an Intel Emu.

  • davm

    KEEP is not intended to be an universal emulator at all but rather a framework to accomodating all the existing and future emulators… so it’s nothing like duplicating effort and the fact that it’s in java has no impact whatsoever on performance since emulators are incorporated as they are.

    This article is basically wrong and would need to be deleted or updated urgently

  • Frameworks do exist right now, indeed. There are MAME & MESS, for instance, and from my standpoint there is no need of some sort of uber-fanta-mega-framework to rule them all.

    And maybe you didn’t know, but Java is crappy to run highly complex and CPU-demanding code, period. Furthermore, MAME and MESS have been developed with archiving in mind. Many, many years ago, with many, many years of precious reverse engineering work that won’t end up in KEEP. So KEEP is a waste of time.

    You are trying to convince me to delete or update the article? Please try harder….

  • @davm

    http://www.keepproject.eu/ezpub2/index.php?/eng/About-KEEP

    Some select snippets:

    > KEEP (Keeping Emulation Environments Portable) will develop an Emulation Access Platform
    > developing flexible tools for accessing and storing a wide range of digital objects.
    > KEEP will address the problems of transferring digital objects stored on outdated computer media such as floppy discs onto current storage devices.
    > the project will deliver understanding about how to integrate emulation-based solutions with an operational electronic deposit system.

    It sounds to me like they are intentionally duplicating efforts.

    You said:
    > a framework to accomodating [sic] all the existing and future emulators

    Are you referring to this?:

    > KEEP will develop an Emulation Access Platform to enable the accurate rendering of these objects

    The language of this document – arguably the most important public-facing document – seems to be deliberately vague about being a launcher for existing emulators.

    It would be more useful for them to get a human being to rewrite their ‘about’ and to write a proper manifesto which clearly details what they actually want to achieve and why they can provide something new which nobody else already has or will.

  • http://www.keepproject.eu/ezpub2/index.php?/eng/About-KEEP/Emulation-Access-Platform/KEEP-Emulation

    Aah, I see they do explicitly list a section for “emulators”, but this still doesn’t explicitly say they’re using emulation projects external to themselves.

    http://www.keepproject.eu/ezpub2/index.php?/eng/About-KEEP/Technical-solution

    That page also gets a bit more technical, but it still doesn’t talk about not duplicating efforts. It’s talking as though all those issues weren’t solved long ago.

  • zoboglobo

    Thinking that all people working in this project are just “european academicians” having “no intention to dirty their hands with the hobbyists’ work” seems a bit contemptuous, even if I agree that the content of their website would need serious improvements…

    This european project combines several people coming from various horizons and countries, with different opinions, some of them working for private companies or national institutions that sometimes have their own idea about how all of this should be leaded. So I think I can understand why the echoes we have are a bit fuzzy and/or messy sometimes…

    However, don’t get fooled : some of these people used to have dirt on their hands and they do not spit on hobbyists’ work, as they belong to these hobbyists for a long time now. Even if the final result of KEEP project may not necessarily be as ambitious as some expected, it may directly or indirectly open the way for new technologies for a better management of the bunch of digital data that pile up in national libraries. It may also help hobbyists having their own emulators being recognized by national institutions (and not re-done from scratch ! On what elements such statements are based ?!).

    So I think a more qualified point of view may be necessary waiting for the final result of the project to be known.

    remark: by the way, thinking that the project targets as using Java as an emulation platform is another crazy statement…

  • We’ll see what KEEP will end up being. Meanwhile I can’t help but looking at how much the “hobbyists”‘ emulators are improving right now and how this KEEP project continues to be just a theory.

Leave a Reply

Your email address will not be published. Required fields are marked *