Mechanically Separated Meat

NON-PROGRAMMERS READ THIS

Here, download this silly physics toy:

“Typewriter”

• Mac version
• Windows version
  (Linux users: Source below)

Controls: Keyboard, mouse, F1, F4, ESC

PROGRAMMERS READ THIS

When I started writing Jumpman, something that frustrated me was that there are a couple of seemingly basic things that SDL/OpenGL doesn’t actually provide out of the box, and that I couldn’t seem to find a really good source for sample code for– things like drawing text, or creating a simple GUI, or building a crossplatform binary. So once the game was done I decided to clean up my code a little, strip out the “Jumpman” parts and release the basic skeleton as open source sample code. Below is that code, and a small tutorial on setting up Mac OS X such that it can build Windows and Linux executables. The hope is to make an overall package that would allow someone starting an SDL/OpenGL game to just sit down and start writing, rather than having to spend time downloading and fiddling with libraries.

The Jumpcore package comes in two versions. A minimal version that includes only:

• The ability to draw text (provided by the Freetype and FTGL libraries).
• A code snippet for managing “internal files” (which live in a directory named “Internal” on windows/linux, and inside the application package in OS X)
• Alt-tab support for OS X (SDL does not do this out of the box for fullscreen apps)
• Makefiles (and one .xcodeproj) for Windows, Mac and Linux

And a more fully featured version that also comes packaged with:

• The Chipmunk 2D physics library
• The LodePNG library (and a code snippet for loading PNGs into OpenGL textures)
• The TinyXML library
• Some color conversion routines
• A minimal “ControlBase” GUI library (dependent on Chipmunk)
• The “Typewriter” demo code linked at the top of this post.

The included libraries were picked in an attempt to include all the basic stuff a game needs, while still making the package as easy as possible to port and reuse in weird situations: all the libraries are self-contained and except for SDL itself can be built from the package as source where necessary; nothing depends on anything more complicated than the STL– I avoided heavyweight dependencies like Ogre or libpng; and everything is under a BSD-like license. The biggest limitation of the package at the moment is that it’s a bit mac-centric (I have not tested it with Visual Studio or Dev-C++).

Basically, here’s a box full of Legos and half a robot. Have fun.

DOWNLOAD

• Jumpcore source code (full version w/ typewriter demo)
• Jumpcore source code (minimal version)

HOW TO BUILD

Included is a Jumpcore.xcodeproj for compiling on mac, which can be compiled with XCode; windows makefile and support files are in a folder named win/, and can be compiled with mingw; Linux makefile and support files are in a folder named lin/, and can be compiled with gcc. More detailed instructions for all three platforms follow:

If you’re on a mac:

To build a mac executable, from a mac: Included is a Jumpcore.xcodeproj for use with XCode; just build that in Release mode and it should produce a 10.3.9-compatible universal binary (though note, I’ve not specifically tested it with 10.3.9).

If you’re on a mac and you want to build a Windows executable:

Here’s the best way I’ve found to do this:

1. There is a “Cross Compilers for Mac OS X” page here that actually has OS X installers for mingw. PPC and Intel versions are included; I installed 4.3.0 for Intel. The only problem with these particular installers is they install into strange places, so whichever installer from that page you pick, write down the “Installation directory” listed to the right of it.
2. Once you’ve installed an installer from that page, you need to install SDL headers. In order to do this, go to the SDL download page and look under “Development Libraries” -> “Win32″ -> “Mingw32″. Download that tarball. Once you’ve downloaded it ignore the “INSTALL” file, which is full of lies, and do this: Edit the “Makefile” in the directory so that the “CROSS_PATH” on line 4 is the “Installation directory” you wrote down in step 1. Like in my case this would be:
   CROSS_PATH := /usr/local/i386-mingw32-4.3.0

   Once you’ve done this, run “sudo make cross” and it will install the SDL headers into your mingw directory.

3. Go into the “win/” directory. Run “make” with the argument MINGW=[Installation Directory], where [Installation Directory] is again the directory from step 1– in my case this would be
   make MINGW=/usr/local/i386-mingw32-4.3.0

A directory named “Jumpcore” will be created with a Jumpcore.exe and all the support files necessary.

If you’re on a mac and you want to build a Linux executable:

Just distribute source. No, really. Building Linux binaries for distribution is tricky, and binaries aren’t what people want anyway. However if you want to do what I did and chicken out, what I recommend is installing Virtual Box or Q (probably Virtual Box, though Q is what I used) and loading up an Ubuntu install CD. This is quicker and easier than trying to set up a cross compile. Then go into the “lin/” directory and type “make”.

If you’re on Windows:

I was able to successfully compile Jumpcore on Windows by doing the following:

1. Download and install MinGW. (Make sure to install the C++ package.)
2. Download and install MSYS (it’s part of MinGW, but a separate download)
3. As described on the MinGW install HOWTO, add C:\MinGW\bin to your path: right-click “My Computer”, click “Advanced”, click “Environment Variables”, double-click the line that says “PATH”, and in the second line add the characters ;C:\MinGW\bin
4. Go to the SDL download page and look under “Development Libraries” -> “Win32″ -> “Mingw32″. Download that tarball and open up its contents in MSYS. Type “make native” and it will install itself.
5. A kind of odd step: right-click the file “README.txt”, open it in Wordpad, and immediately save it. (This will strip out my evil UNIX newlines.)
6. Go into the directory win/ and run: make WINDOWS=1

This will create an install directory named “Jumpcore”. If you want to compile for debugging, in that last step type: make WINDOWS=1 DEBUG=1

If you’re on Linux:

Install Freetype and SDL. Go into the directory lin/ and run make. This will create an install directory named “Jumpcore”. If you want to compile for debugging, instead type: make DEBUG=1

GETTING STARTED

Once you get the thing built, you’re going to want to start focusing on swapping out the Typewriter code for your own code. Jumpcore consists of a main.cpp that does basic bringup/teardown and event loop work hopefully good enough for most games, and makes callbacks as appropriate into a display.cpp (display logic) and a program.cpp (game logic) you provide. You’ll want to implement the following methods:

In display.cpp

display_init() - This is called once each time the display surface is initialized. It’s a good place to do things like initialize fonts and textures. (Note it could be called more than once if the window size ever changes.)

display() - This is called when it is time to draw a new frame.

audio_callback() - This is set up as the SDL audio callback.

drawButton (”full version” only) - This is a cpSpaceEach callback which the default display() calls on each interface item. If you want to change the appearance of the ControlBase controls this is a good place to do that.

In program.cpp

program_init() - This is called once when the program begins.

program_update() - The default display() calls this once per framedraw.

program_eventkey() - This is called when SDL gets a key event.

program_eventjoy() - This is called when SDL gets a joystick event.

program_eventmouse() - This is called when SDL gets a mouse event.

program_interface() - This is called after the event system finishes dispatching events to ControlBase controls, to give the interface a chance to redraw itself.

BackOut() - Called when ESC is hit (quits).

AboutToQuit() - Called right before the program quits.

Documentation for the individual libraries and functions included with Jumpcore can be found on these separate pages:

• Basic Jumpcore functions
• ControlBase GUI library
• Chipmunk documentation
• LodePNG home page
• TinyXML documentation

LIMITATIONS AND POSSIBLE FUTURE IMPROVEMENTS

I’m not really sure if this is ultimately going to be useful to anyone, and I don’t intend to maintain it unless there are people actually using it. However if there turns out to be any interest in this there are a few things I’d like to improve in a future release:

• The package contains everything you need to build a Windows version from a Mac. It would be awesome if I could eventually reach the point where a Windows user could build a Mac version (is that even possible?).
• Linux version is poorly tested in general. I have reports of issues on 64 bit systems, and the original Jumpman code seemed to have issues with switching to and from full screen mode.
• The final executable size is pretty large– 2 or 3 MB compressed for the very minimal typewriter demo. I’m curious if this can be improved on. At least on the mac a large chunk of this is taken up by SDL, which gets bundled along with the executable. However, to someone who’s using OpenGL to draw, a lot of this is wasted space– because much of the complexity in SDL is taken up by the 2D drawing support. I’d like to try to swap out the SDL libraries for versions that lack 2D drawing.
• iPhone compatibility? Now that I’m doing iPhone development I’m becoming pretty firmly convinced it does not make sense to create a single codebase that compiles on both PC and iPhone– the platforms are too different– but maybe it would make sense to rewrite some parts of the typewriter demo to make portability to something like iPhone easier (for example, rewriting the drawing code to be OpenGL ES-compatible).
• I am not sure that every library included with this is the most recent version.
• The one “every game needs this” feature that isn’t in this package is configurable joystick/gamepad support. I’m not sure whether it makes sense to try to add it or not.

Finally, there have actually been a number of interesting-looking SDL “game engines” released lately, so you should be aware of those in case one fits your needs better than Jumpcore does. Two I’m aware of are 2D Boy’s Boy engine and Mattias Gustavsson’s Pixie engine (feel free to share more in the comments below).

So my Jumpman game has a level editor, and I thought I should make a place for people to put the level packs they’ve made.

Until I think of a better way to do this: if you made a level pack you want to share, zip up the .jmp and email it to jumpmanlevels@gmail.com and I’ll post it here. Include the name you want me to identify you by, and (if you want) a URL to link you at. (Notes: By emailing me your level pack, you are of course giving me permission to mirror it here. I reserve the right to not post your level pack, or to omit your URL, or to write whatever description seems appropriate.)

If you want to play anything below, just download the file into the same folder as the Jumpman program and unzip it. Then run Jumpman and click “Editor”.

All the level packs below are under 15 kilobytes. For calibration on the times, I can beat the main Jumpman level pack in about 15 minutes (…I’ve had practice).

Enjoy:

Peter Mawhorter’s Levels
(Operation Jumpman, Jumpquest, Pixels, Slalom, Uneven)
Peter has made an entire series of excellent level packs which you can download at his website above.

Freefall
Creator: mcc | Type: Playground | Added: 3-22-09
A little experiment I made which I think you will find interesting. It’s only one level long but it may take you awhile. There is no exit, your goal is to reach the orange ball.

Precaution
Creator: fzeroracer | Type: Cruel Challenge | Added: 3-1-09
Do you think Jumpman is hard? Jumpman is not hard. This is hard.

Sonic
Creator: Fly | Type: Playground | My time: 0:26, 0 lives
Fly is from Chile. I assume the “Sonic” burger chain must be very popular in Chile, since I cannot think of anything else this name could possibly be in reference to.

JumpPAC
Creator: ToastyKen | Type: Platformer | Added: 3-1-09 | My time: 5:26, 23 lives
A collection of platforming challenges in strangely familiar settings.

Mikesta
Creator: Mikesta | Type: Cruel Challenge | Added: 3-1-09 | My time: 26:41, 152 lives
Mikesta takes the editor’s edge cases and does something truly epic with them.

Jumpman Gaiden (beta)
Creator: Kitsune Zeta | Type: Platformer | Added: 3-3-09 | My time: 3:26, 5 lives
This is the guy who’s been posting in the comments saying that he is trying to create a level pack that can serve as a full replacement for the Jumpman main level set, complete with tutorial levels. The version linked here is only like 15 levels but it seems to be going well. The sense of visual style is fantastic, the early bits look, like… art deco, somehow, which I’m not sure how pixel art can even be art deco but it seems to work?

Easy
Creator: AndrewFM | Type: Platformer | Added: 3-3-09 | My time: 3:26, 5 lives
A fairly basic level set but with some very creative ideas in places. Warning, it appears to be possible to get permanently stuck in level 13.

Randomness
Creator: Zachary Nicholas | Type: Platformer | Added: 3-22-09 | My time: 3:28, 30 lives
A short series of platforming challenges.

Kyou’s Playground
Creator: Kyou | Type: Playground | Added: 3-1-09 | My time: 0:57, 11 lives
Kyou has some fun with the physics engine… maybe a little too much fun.

Farik
Creator: Farik | Type: Short Platformer | Added: 3-1-09 | My time: 3:50, 1 life
A neat little combination of platforming level and pixel art.

Fly
Creator: Fly | Type: Short Platformer | Added: 3-1-09 | My time: 0:56, 14 lives
Fly does not see what’s so great about platforms that stay in one place.

FAN ART

A man named JB Winter sends in this awesome hypothetical Jumpman box art. I can totally imagine seeing this at Babbage’s.

Also:

I made a video game.

DOWNLOAD (Version 1.0.2)

• Windows 2.1 MB
  
• Macintosh 3.3 MB (Universal Binary)
• Linux 0.3 MB (x86 binary, requires SDL and Freetype)

GAMEPLAY VIDEO

FEATURES

• Old-school puzzle platforming with some twists
• Low-definition graphics
• Gamepad support
• Full level editor

PLOT

• Guide Jumpman to the exit.

CLICK HERE TO JUMP TO THE PRETTY COLOR-CODED FULL RESULTS

I’m just gonna copy and paste the explanation I gave last year:

For the last few years I’ve been hosting this Game of the Year poll for the users of some forums I read. There are a lot of GOTY polls out there, but this one I think is kind of special. Most polls, you’re given a list of four or five options and you’re asked to pick the one you liked best. This poll, people are given a list of a couple of hundred options, consisting of every new game released in the previous year– and asked to rate their top ten or twenty.

This does a few interesting things. First off, we get to see all the information about what people’s second, third etc choices are. Second off, because the second, third etc choices count, people are more likely to vote for the game they want to win, rather than the game they think is likely to win– they’re less likely to engage in “strategic voting”. Finally, because we have all this information, we’re actually able to provide somewhat reasonable rankings for something like the top hundred or so games of last year.

The full results– showing the exact number of voters who ranked each game first, second, third place etc– can be found here. In the meantime, the final results were:

1. Fallout 3 (8780) *** GAME OF THE YEAR ***
2. Left 4 Dead (6626)
3. Grand Theft Auto 4 (5032)
4. Super Smash Bros. Brawl (4321)
5. Rock Band 2 (3290)
6. Dead Space (3151)
7. Gears of War 2 (2942)
8. Fable 2 (2751)
9. Braid (2729)
10. Metal Gear Solid 4 (2666)
11. Little Big Planet (2520)
12. No More Heroes (2241)
13. Audiosurf (2152)
14. Castle Crashers (2083)
15. Valkyria Chronicles (2027)
16. Mario Kart Wii (2014)
17. The World Ends with You (2000)
18. World of Warcraft: Wrath of the Lich King (1914)
19. Penny Arcade Adventures: On The Rain-Slick Precipice of Darkness Ep. 1 (1910)
20. Sins of a Solar Empire (1850)

The numbers in parentheses are the final scores each game got under the poll’s ranking system. (The scores in general were a lot closer than last year–basically all the rankings 14-18 are within a couple votes of each other!) Thanks if you voted, and some more elaborate analysis of the results (plus an explanation of the scores) can be found below.

NOTEWORTHY WINNERS

• GOTY 2008:
  

  #1, Fallout 3

• Top-ranked Wii Exclusive:
  

  #4, Super Smash Bros. Brawl

• Top-ranked 360 Exclusive:
  

  #7, Gears of War 2

• Top-ranked PS3 Exclusive:
  

  #10, Metal Gear Solid 4

• Top-ranked PC Exclusive:
  

  #13, Audiosurf

• Top-ranked DS Exclusive:
  

  #17, The World Ends With You

• Top-ranked PSP Exclusive:
  

  #39, Crisis Core: Final Fantasy VII

• Best FPS:
  

  #2, Left 4 Dead

• Best RPG:
  

  #1, Fallout 3

• Best Sports Game:
  

  #27, Burnout Paradise

• Best Game Only Available Through A Console Download Service:
  

  #8, Braid

• Special “Cult” Award (see below):
  

  #26, Persona 4 & #15, Valkyria Chronicles (Tie)

NOTEWORTHY LOSERS

• Best game of 2008 which somehow nobody considered to be their #1 pick: #33, Spore
• Worst game of 2008 that at least one person considered their #1 pick: #179, Midnight Club: Los Angeles (Only two people voted for this)
• Worst game of 2008: #203, Mystery Case Files: MillionHEIR (Only one person voted for this; it was their #20 pick)

There were also ten games which were listed, but which no one voted for at all.

ALTERNATE SCORING METHODS

The rankings listed above are based on what was intended to be an approximation of Condorcet voting, but which I’m told is actually closer to the Borda count. In my Borda-ish voting method, each vote cast for a game gives that game a certain number of points. If someone ranks a game #1, that game gets 20 points. If they rank it #2, the game gets 19 points. If they rank it #3 the game gets 18 points… and so on. I have a script that checks a couple of alternate ways of ranking the same data, though.

For example, if we rank games only by the number of first post votes they got, we get a wildly different list:

First Past the Post

1. Fallout 3 (182 first-place votes)
2. Left 4 Dead (109)
3. Super Smash Bros. Brawl (42)
4. Metal Gear Solid 4 (42)
5. Valkyria Chronicles (39)
6. Persona 4 (39)
7. Grand Theft Auto 4 (35)
8. Dead Space (31)
9. Rock Band 2 (30)
10. The World Ends with You (30)
11. World of Warcraft: Wrath of the Lich King (30)
12. Gears of War 2 (23)
13. Little Big Planet (21)
14. No More Heroes (19)
15. Braid (15)
16. World of Goo (14)
17. Spelunky (12)
18. Sins of a Solar Empire (11)
19. Fable 2 (10)
20. Prince of Persia (10)

Every year when I do this there’s some game which scores horribly low in the objective rankings but gets a really startling proportion of first-place votes; last year the standout game in the “cult” department was Persona 3; this year the standout was, interestingly enough, Persona 4, which only got 87 votes at all, placing it at #26 in the overall rankings– but nearly half of those votes, a full 39, ranked it in first place, putting it in sixth place in the First Past the Post ranking above. Tying Persona 4 in the First Past the Post ranking is Valkyria Chronicles, which did a little better in terms of how many people voted for it (117 votes) but which still gets a pretty great cult ranking since one in three of those voters considered it their #1 game. (Honorable mention in the cult category should probably go to “Spelunky“, a wildly obscure but kind of awesome freeware pixel art game released in the last two weeks of December, which came in way down at 51st place in the overall rankings but managed to come in 17th in first-pace votes– with again nearly one-third of the people who voted for Spelunky at all rating it #1.)

I also did two more ways of sorting the rankings: an “approval” vote, where nothing is counted except the number of votes a game received (i.e. a first-place and a twentieth-place ranking count the same– all the matters is if the game was on someone’s list); and an instant runoff vote. Most years I’ve done this the Instant Runoff and pseudo-Borda rankings have been almost the same, but this time there were some interesting differences (with the biggest one being, for some reason I don’t understand, World of Goo somehow jumping a good seven spots in the rankings?!). Your eyes are probably starting to glaze over at this point, so I bolded the places where these two votes differ from the normal rankings:

FINALLY: PER-FORUM BREAKDOWNS

As mentioned before, this poll mostly exists for a handful of video game forums where some people I know post. Since last year when I started posting the results on this blog, I’ve tried to actually run some extra results, in each case counting only those voters who– as far as one could tell from looking at the logs– had come to the poll from one particular forum or other.

So, here you have it– these numbers aren’t totally accurate because my logging method is not entirely trustworthy, but here’s an approximate by-forum breakdown of these results. Links go to color-coded full listings.

Short version: Just watch this video.

Okay, now what was that?

So a few months back some of my friends were passing around these videos of something called “Kaizo Mario World“, which I was told, at first, translated to “Asshole Mario World”. This turned out to have actually been a misunderstanding of something in the youtube posting of the original creator’s videos:

[Asshole Mario] is not the real name for this series of videos, but it is my personal name for it.
The literal translated name for 自作の改造マリオ（スーパーマリオワールド）を友人にプレイさせる is “Making my friend play through my own Mario(Super Mario World) hack”, hence Kaizo(hack) Mario to the USA.

…but, the name is pretty appropriate. Kaizo Mario World is one of a series of rom hacks people create in special level editors that let you take Super Mario World and rearrange all the blocks; the point of Kaizo appears to have been to create the most evil Super Mario World hack ever.

I started watching these videos, but after seeing how the player got past the first gap stopped, went wait, this actually doesn’t look so bad, and started playing it instead. It’s actually not that bad! I was expecting it to be like Super Mario Frustration, Kaizo Mario World’s equivalent in Super Mario Bros. 1 hacks– all ridiculous jumps that require pixel-perfect timing, memorizing the location of a bunch of hidden blocks that exist only to foil a jump and, occasionally, actually exploiting glitches in the game engine.

Kaizo Mario World actually turns out though really to be kind of more like a puzzle game– giving you a series of seemingly impossible situations and then leaving you to figure out how to get past them. It only uses the sadistic-invisible-block trick sparingly (and, hey, even SMB2JP did that a couple times). And it actually turns out to be kind of fun.

It’s still sadistically hard, though, so if you want to play it you have to use what are called “save states”. Most emulators let you do this kind of save-and-rewind thing, where if you screw up you can back up just a few seconds to the last time you were standing in a safe place. So if you’re playing Kaizo Mario world you find yourself playing the same four-second section over and over and over until you get the jump just right, listening to the same two seconds of the soundtrack looping Steve Reich style…

Anyway, the idea for the video up top was inspired by an offhanded comment in the “original” Kaizo Mario World youtube post I linked above:

The original videos were in god awful codecs that were a bitch to convert, so unfortunately the Tool Assisted Speedruns came first to most youtube watchers.
This is rather unfortunate, as I feel you lose a lot of the “appeal” by watching those.

This refers to the way that most emulators, if you are recording a video of yourself playing a game and you do the save-state rewind thing, they’ll rewind the video too, such that the video shows only your final attempt, not any of your messups. People use this to make “speedruns” showing a best-of-all-possible-worlds recording of them playing through some game or other, with all the errors scrubbed out. The guy’s point was that watching Kaizo Mario World this way kind of ruins it, since most of what makes Kaizo great is watching someone fail over and over and over again until they finally get it right.

On the other hand, Kaizo Mario World involves SO much failing that this means all the “real” videos are, like, twenty minutes long just to get through what in a tool-assisted run would have been a two-minute level. So I was thinking, what if you had a special tool that instead of erasing all the screwups, it saved all of them and made a video of all the screwups plus the one successful path superimposed? I kept thinking about this and eventually I just sat down and hacked SNES9X to work exactly like that. The result was the video up top, showing the 134 attempts it took me to successfully get through level 1 of Kaizo Mario World.

I think I’m going to make some more videos in this style of different Kaizo Mario World levels and post them back here, but in the meanwhile, if you want to make your own many-worlds speedrun videos, here’s my custom version of SNES9X 1.43 with the multi-record function:

1. For the Mac OS X version, click here.

2. For a Windows version, click here. (Many thanks to Syndalis of 360Arcadians for compiling this for me.)
3. If you want a Linux version, you’ll have to compile that yourself, but you can do this by finding a copy of the 1.43 source and replacing movie.cpp with this.
4. And for the full Mac OS X source, click here.

[Update 2/9/08: The Mac version now correctly processes movies recorded in the Windows version.]
[Update 2/10/08: Mac version updated to fix a problem where certain kinds of corrupt recording files could cause the program to loop endlessly; window titlebar now contains status information.]

Note that this is a quickly-tossed-together hack all done to make a single video, and I make NO promises as to the quality, ease-of-use, correctness, or safety of these links. Also, I think the video feature should work with any SNES game, but I’ve only tested it with Kaizo. If anyone attempts to try this yourself, I’d be curious to hear about your results.

To make a video: First, use SNES9X’s “record movie” function to record yourself playing some game; while the game is running, use the save and restore feature at least once. When you’re done, you’ll find that SNES9X has created a yournamehere.smv file and also a series of files with names like yournamehere.smv.1, yournamehere.smv.2, etc. These .number files are all the different “mistake” playthroughs, so keep all these files together in one directory.

To turn this into an actual movie you can watch, you will need to use the OS X version of the emulator. Unfortunately, the Windows and Linux versions can only record multiple-run SMVs– they can’t do the export-to-quicktime thing. The quicktime-export code is based on alterations to the mac-specific parts of 1.43 (although considering that I hear the Quicktime API is mostly identical between Mac and Windows, it might be pretty easy to port that code to Windows at least…).

Anyway, in the OS X version, open up the appropriate ROM and choose “Export to Quicktime Movie” from the Option menu. Before leaving the export dialogue, make sure to click the “Compression…” button. You *MUST* choose either the “None” or “Planar RGB” codecs, and under the “Compressor” pane you *MUST* choose a depth of “Millions of Colors+”. The “+” is important. Once you’ve saved the movie location, go to “Play Movie” in the Option menu and choose the .smv you want to play. The emulator will play through each of the playbacks one by one; when it’s done (you’ll know because the background turns back on) your movie will appear in the location you chose. Note that there’s one more step! You won’t be able to actually play this movie, at least not very well, because the export feature works by creating a different movie track for each playthrough and the file will be huge and bloated. Open your video in Quicktime Player, then choose “export” and export to some video codec with actual compression (like H.264). This will flatten all the different layers of the movie into one. Okay, NOW you’re done.

…So what’s this about quantum physics? Oh, right. Well, I kind of identify the branching-paths effect in the video with the Everett-Wheeler “Many Worlds Interpretation” of quantum physics. Quantum physics does this weird thing where instead of things being in one knowable place or one knowable state, something that is quantum (like, say, an electron) exists in sort of this cloud of potentials, where there’s this mathematical object called a wavefunction that describes the probabilities of the places the electron might be at a given moment. Quantum physics is really all about the way this wavefunction behaves. There’s this thing that happens though where when a quantum thing interacts with something else, the wavefunction “collapses” to a single state vector and the (say) electron suddenly goes from being this potential cloud to being one single thing in a single place, with that one single thing randomly selected from the different probabilities in the wavefunction. Then the wavefunction takes back over and the cloud of potentials starts spreading out again from that randomly selected point.

A lot of scientists really don’t like this “collapse” thing, because they’re uncomfortable with the idea of nature doing something “at random”. Physics was used to dealing with randomness before quantum physics came along– the physics of gases are all about the statistics of randomly moving gas particles, for example– but those kinds of randomness aren’t assumed to be actually random, just “effectively random” because the interactions of air molecules are so chaotic and complicated that they’re too unpredictable for humans to track. Think about what happens when you roll a die: the number that comes up when the die lands isn’t strictly speaking “random”, it’s absolutely determined by the physics of motion and the velocity at which you let go of the die and so forth. The “randomness” of a die roll isn’t about actual indeterminacy, but rather just a way of talking about your ignorance of how the deterministic processes that control the die operate. Quantum physics, on the other hand, has things that as far as anyone can tell are really, objectively random, with no mechanism producing that randomness and nowhere apparent to stick one.

Since this makes some physicists uncomfortable, they came up with a sort of a philosophical trick: they interpret quantum physics in such a way that they say when there’s more than one possible random outcome of some quantum process, then the different possibilities all happen, in alternate universes. They can’t prove or disprove that this idea is true– from the perspective of someone inside one of these universes, everything behaves exactly the same as if the “wavefunction collapse” really was just picking a random option. But it’s one way of looking at the equations of quantum mechanics, and as far as the mathematics cares it’s as valid as any other. Looking at things this way, if there’s a 3/4 chance of a quantum process doing one thing and a 1/4 chance of it doing the other, then we get three universes where the one thing happens and one universe where the other one does. This does mean that there’s some universe where two seconds ago all of the atoms in your heart spontaneously decided to quantum-tunnel two feet to the left, but in almost every universe this doesn’t happen so we don’t worry about that.

Science fiction authors love this. There’s a bunch of stories out there exploring this idea of a multiverse of infinite possibilities all occurring side by side (the best of these I’ve ever read being Robert Anton Wilson’s Schrödinger’s Cat). Most of these stories get things totally wrong. Science fiction authors like to look at many-worlds like, this morning you could either take the bus to work or walk, so the universe splits in two and there’s one universe where you decided to walk and one universe where you decided to take the bus. This is great for purposes of telling a story, but it doesn’t really work like that. The many-worlds interpretation is all about the behavior of quantum things– like, when does this atom decay, or what angle is this photon emitted at. Whereas human brains are big wet sloppy macroscopic things whose behavior is mostly governed by lots of non-quantum processes like neurotransmitters releasing chemicals.

This said, tiny quantum events can create ripples that have big effects on non-quantum systems. One good example of this is the Quantum Suicide “experiment” that some proponents of the Many-Worlds Interpretation claim (I think jokingly) could actually be used to test the MWI. The way it works is, you basically run the Schrödinger’s Cat thought experiment on yourself– you set up an apparatus whereby an atom has a 50% chance of decaying each second, and there’s a detector which waits for the atom to decay. When the detector goes off, it triggers a gun, which shoots you in the head and kills you. So all you have to do is set up this experiment, and sit in front of it for awhile. If after sixty seconds you find you are still alive, then the many-worlds interpretation is true, because there is only about a one in 10¹⁸ chance of surviving in front of the Quantum Suicide machine for a full minute, so the only plausible explanation for your survival is that the MWI is true and you just happen to be the one universe where the atom’s 50% chance of decay turned up “no” sixty times in a row. Now, given, in order to do this, you had to create about 10¹⁸ universes where the Quantum Suicide machine did kill you, or copies of you, and your one surviving consciousness doesn’t have any way of telling the people in the other 10¹⁸ universes that you survived and MWI is true. This is, of course, roughly as silly as the thing about there being a universe where all the atoms in your heart randomly decided to tunnel out of your body.

But, we can kind of think of the multi-playthrough Kaizo Mario World video as a silly, sci-fi style demonstration of the Quantum Suicide experiment. At each moment of the playthrough there’s a lot of different things Mario could have done, and almost all of them lead to horrible death. The anthropic principle, in the form of the emulator’s save/restore feature, postselects for the possibilities where Mario actually survives and ensures that although a lot of possible paths have to get discarded, the camera remains fixed on the one path where after one minute and fifty-six seconds some observer still exists.

Note: Please do not use the comments section of this post to discuss ROMs or where to get them. IPSes are okay. Thanks.

CLICK HERE TO JUMP TO THE PRETTY COLOR-CODED FULL RESULTS

So for the last few years I’ve been hosting this Game of the Year poll for the users of some forums I read. There are a lot of GOTY polls out there, but this one I think is kind of special. Most polls, you’re given a list of four or five options and you’re asked to pick the one you liked best. This poll, people are given a list of a couple of hundred options, consisting of every new game released in the previous year– and asked to rate their top ten or twenty.

This does a few interesting things. First off, we get to see all the information about what people’s second, third etc choices are. Second off, because the second, third etc choices count, people are more likely to vote for the game they want to win, rather than the game they think is likely to win– they’re less likely to engage in “strategic voting”. Finally, because we have this information, we’re actually able to provide somewhat reasonable rankings for something like the top hundred or so games of last year.

The full results– showing the exact number of voters who ranked each game first, second, third place etc– can be found here. In the meantime, the final results were:

1. Portal (9532) *** GAME OF THE YEAR ***

2. Bioshock (8004)
3. Super Mario Galaxy (7968)
4. Mass Effect (5874)
5. Team Fortress 2 (5256)
6. Call of Duty 4 (5051)
7. Halo 3 (4848)
8. Half Life 2: Episode 2 (4660)
9. Rock Band (3788)
10. Guitar Hero 3 (2968)
11. Metroid Prime 3: Corruption (2948)
12. Assassin’s Creed (2937)
13. Legend of Zelda: Phantom Hourglass (2605)
14. World of Warcraft: Burning Crusade (2324)
15. Super Paper Mario (2215)
16. Pokemon Diamond/Pearl (2083)
17. Crackdown (1978)
18. Puzzle Quest: Challenge of the Warlords (1773)
19. Phoenix Wright: Ace Attorney - Justice for All (1621)
20. Zack and Wiki (1453)

The numbers in parentheses are the final scores each game got under the poll’s ranking system. (Bioshock and Galaxy were close!, as were Guitar Hero, Metroid, and Assassin’s Creed.) Thanks if you voted, and some more elaborate analysis of the results (plus an explanation of the scores) can be found below.

NOTEWORTHY WINNERS

• GOTY 2007:
  

  #1, Portal

• Top-ranked 360 Exclusive:
  

  #4, Mass Effect

• Top-ranked Wii Exclusive:
  

  #3, Super Mario Galaxy

• Top-ranked PS3 Exclusive:
  

  #34, Uncharted: Drakes Fortune

• Top-ranked PC Exclusive:
  

  #14, World of Warcraft: Burning Crusade

• Top-ranked DS Exclusive:
  

  #13, Legend of Zelda: Phantom Hourglass

• Top-ranked PSP Exclusive:
  

  #56, Castlevania: Dracula X Chronicles

• Top-ranked GBA Exclusive:
  

  #100, Legend of Spyro: Eternal Night

• Best FPS:
  

  #1, Portal

• Best RPG:
  

  #4, Mass Effect

• Best Sports Game:
  

  #27, Forza Motorsport 2

• Best Game Only Available Through A Console Download Service:
  

  #49, Sin and Punishment

• Special “Cult” Award (see below):
  

  #25, Persona 3

NOTEWORTHY LOSERS

• Best game of 2007 which somehow nobody considered to be their #1 pick: #19, Phoenix Wright: Ace Attorney - Justice for All

• Worst game of 2007 that at least one person considered their #1 pick: #187, Hammerfall (Only two people voted for this)
• Worst game of 2007: #208, SNK vs Capcom Cardfighters DS (Only one person voted for this; it was their #19 pick)

There were also five games which were listed, but which no one voted for at all.

ALTERNATE SCORING METHODS

The rankings listed above are based on an approximation of Condorcet voting. In my pseudo-Condorcet approximation, each vote cast for a game gives that game a certain number of points. If someone ranks a game #1, that game gets 20 points. If they rank it #2, the game gets 19 points. If they rank it #3 the game gets 18 points… and so on. I have a script that checks a couple of alternate ways of ranking the same data, though.

For example, if we rank games only by the number of first post votes they got, we get a slightly different list:

First Past the Post

1. Portal (176 first-place votes)
2. Super Mario Galaxy (170)
3. Mass Effect (105)
4. Bioshock (88)
5. Rock Band (48)
6. Team Fortress 2 (44)
7. Call of Duty 4 (39)
8. Halo 3 (27)
9. Persona 3 (20)
10. World of Warcraft: Burning Crusade (14)
11. Metroid Prime 3: Corruption (12)
12. S.T.A.L.K.E.R.: Shadow of Chernobyl (10)
13. Half Life 2: Episode 2 (10)
14. Zack and Wiki (9)
15. Uncharted : Drakes Fortune (9)
16. Phoenix Wright: Ace Attorney - Trials and Tribulations (8)
17. Forza Motorsport 2 (7)
18. Legend of Zelda: Phantom Hourglass (6)
19. Pokemon Diamond/Pearl (6)
20. skate. (5)

Every year when we do this there’s some game which scores horribly low in the objective rankings but gets a really startling proportion of first-place votes; this year the standout game in the “cult” department was Persona 3, which only got 78 votes at all, placing it at #25 in the overall rankings– but 20 of those votes ranked it in first place, putting it in ninth place above.

I also did two more ways of sorting the rankings: an “approval” vote, where nothing is counted except the number of votes a game received (i.e. a first-place and a twentieth-place ranking count the same– all the matters is if the game was on someone’s list); and an instant runoff vote. Almost every time I’ve ever done this the Instant Runoff and pseudo-Condorcet rankings have been almost the same, but this time they were actually kind of different. Your eyes are probably starting to glaze over at this point, so I bolded the places where these two votes differ from the normal rankings:

FINALLY: PER-FORUM BREAKDOWNS

As mentioned before, this poll mostly exists for a handful of video game forums where some people I know post. This year, I decided to actually run some extra results, in each case counting only those voters who– as far as one could tell from looking at the logs– had come to the poll from one particular forum or other. Meanwhile, as coincidence would have it, a few days into the vote one of the posts from my blog– where I had also posted about the poll– got linked by Digg, and as far as I can tell from the logs a group of the Digg users actually clicked over to the next post and voted in the poll.

So, here you have it– these numbers aren’t totally accurate because my logging method is not entirely trustworthy, but here’s an approximate by-forum breakdown of these results. Links go to color-coded full listings.

For the last week I’ve been playing this game called “Super Mario Galaxy”, and if it isn’t the best game ever made, then it’s in the top three. It has fanservice by the buckets, and a soundtrack that soars as if the game itself is just happy it is being played, and little blue tractor beam stars that sing you sad little theremin songs, and evil hats. Things happen for no reason, like they used to in old NES video games, and you don’t care why, you just love it. I don’t even know the words to express how much fun I’ve been having with this game.

So instead of trying, I’m going to instead write about the physics problems the game poses.

The gimmick in Super Mario Galaxy is that where previous Mario games had you jumping between platforms floating in the air in the Mushroom Kingdom, the new one has you jumping between little bitty planetoids floating out in space. It works like this:

You get the idea pretty quickly. So here’s what I wonder:

Most of Mario Galaxy is spent running around on the surfaces of those little asteroids, like you see on the video. The asteroids vary in size and shape, although most are roughly spherical and on average each asteroid is about the size of a Starbucks franchise. Every single one of them, regardless of size or shape, has completely normal earth gravity. And it’s hard not to think, when you see Mario taking an especially long jump on a small asteroid and landing about halfway around the planet, that he kinda looks like he’s having a lot of fun. So what I want to know is, is any of this possible in real life?

Would it be physically possible– assuming that you have more or less unlimited resources and futuristic engineering, but are still bound by the laws of known physics– to actually build a bunch of little asteroids like this, with compact volume but earth gravity? Say, if you’re a ringworld engineer or a giant turtle with magic powers. Could you build the Mario Galaxy universe?

Here’s what I worked out:

So first off, clearly we’re not going to be able to do most of the things in the Mario Galaxy universe, since SMG is of course a game and obviously they weren’t trying to be realistic or imitate anything like real physics. I’m going to just ignore these things– for example, I’m ignoring anywhere where there’s just a vanilla gravity field pointing in some direction, as if generated by a machine. As far as we know, that’s just not possible. In the physics we know, the only way to create a gravity field is to put a bunch of mass in the place that you want the gravity to point toward.

This isn’t so bad, since a lot of the planetoids in Galaxy look like this might well be what they are– just a big lump of mass creating a gravity well. In fact, some of the planets– for example the one at the start of the video above– are actually shown to just be thin hollow shells with a black hole at the center. So, in the post that follows, I show what you’d have to do to build one of these planets. I’m going to consider just a single example planet, of about the size of the one from the video above; you’d have to use different masses and densities for each planet in the game, since each has a different size but the exact same amount of gravity at the surface.

ANALYSIS

The first question to ask here is, how much mass do we need? Well, the planet in the video, eyeballing it, looks like it’s about as wide as a 747 is long. Wikipedia says a 747 is 230 feet long.

Newton’s formula for gravity is m_1*a = G * m_1 * m_2 / r^2, where I take m_1 as the mass of Mario and m_2 is the mass of the planetoid. Solving for m_2, I get:

m_2 = a * r^2 / G

r is 115 feet (half a 747), and a is earth gravity, 9.8 m/s/s.

Plugging in to google calculator, I get:

((9.8 m (s^(-2))) * ((115 ft)^2)) / G = 1.8043906 × 10^14 kilograms

So, if you want to get earth-like gravity on the surface of a 230-foot-diameter sphere, you’re going to need about 1.8 * 10^14 kg of mass. This is not that much! Checking Wikipedia I find even the smallest moons and dwarf planets in the solar system get up to about 10^20 kg. in fact, 2 * 10^14 kg is just about exactly the mass of Halley’s Comet. This sounds attainable; other characters are shown hijacking comets for various purposes elsewhere in Mario Galaxy, so no one will notice a few missing.

Meanwhile, Wikipedia’s formula for escape velocity for a planet ( sqrt( 2GM / r ) ) tells us that escape velocity from this planetoid will only be about:

sqrt((2 * G * (1.8043906 × (10^14) kilograms)) / (115 feet)) = 26.2110511 m / s

Which is about 60 MPH. So the cannon stars Mario uses in the video to move from planet to planet should work just fine. (Although platforms might not work so well, at least not tall ones; since the planetoid is so small, you’d only have to get about 45 or 50 feet up off the ground before the amplitude of gravity is halved. Actually gravity will fall off so quickly on the planetoid that there would be a noticeable gravity differential between your feet and your head: a six-foot-tall person standing on it would experience about 1 m/s^2 more acceleration on their feet than their head. So expect some mild discomfort.)

At this point the only question is: If one were to attempt to fit Halley’s Comet into a 230-foot-diameter sphere, would this turn out to be impossible for any reason? In answering, I’m going to consider two cases: One where the mass is in a black hole at the center of the sphere; and one where the mass is distributed through the sphere evenly. In both I’ll try to see if anything really bad happens.

So, first, the black hole case. Looking here I’m told that the event horizon of a mature black hole should be equal to G * M / c^2, and Wikipedia tells me that the Schwarzschild radius (the radius you have to pack a given mass into before the black hole starts to form on its own) is about twice that. So plugging this in, I find that the radius of this black hole at the center of the sphere is going to be:

(G * (1.8043906 × (10^14) kilograms)) / (c^2) = 1.33970838 × 10^-13 meters

… uh oh! Now we seem to be running into trouble. If someone wanted to construct a black hole with the mass of Haley’s comet, they’d somehow have to pack the mass of the whole thing into… well, google claims the diameter of a gold atom is 0.288 nanometers, so… about one-hundredth of the diameter of a gold atom?! That doesn’t sound very feasible.

On the other hand, considering the case where the sphere is solid, one finds that the required density is going to be about:

(1.8043906 × (10^14) * kilograms) / ((4 / 3) * pi * ((115 feet)^3)) = 1.00023843 × 10^9 kg / m^3

As remarkable coincidence would have it, 10^9 kg/m^3 is, according to wikipedia, exactly the density of a white dwarf star, or the crust of a neutron star! So this is sounding WAY easier than the black hole plan: All you have to do is go in and chip off 180,000 cubic feet of the crust of a neutron star, and you’ve got your planetoid right there.

Of course, there’s still one more step. First off, I’m not sure what the temperature of a block of white dwarf matter that size would be, but you might not want to actually walk on it. For another thing, the reason why a white dwarf or the crust of a neutron star has that much density in the first place is that all that matter is being held in place by the incredible gravitational weight of the star the matter is attached to– your average white dwarf is about the size of Earth, which by star standards is tiny, but which compared to our little Mario Galaxy planet is rather large. So if you tore off a 747-sized chunk of one of these stars and just dumped it in space, it would almost certainly explode or expand enormously or something, because the chunk’s gravitational pull on itself would probably not be sufficient to hold it together at the white dwarf density. So we’re going to need some kind of a shell, to hold the white dwarf matter in place and (one assumes) trap things like heat and radiation inside.

When you get to the density a white dwarf is at, the main thing you have to worry about is what’s called degeneracy pressure. Usually, when you try to compress matter, the resistance you meet is due to some force or other– the force holding atoms in a solid apart, for example, or the accumulated force of marauding gas molecules striking the edge of their container. When you get to anything as dense as a white dwarf, though, the particles are all basically touching (”degenerate”), and the main thing preventing you from compressing any further is literally just the physical law that prevents any two particles from being in the same place at the same time, the Pauli Exclusion Principle. If you try to compress past that point, a loophole in the exclusion principle comes into play: it’s actually possible for two particles to share the same chunk of space as long as they’re in some way “different”, for example if one of them is in a higher energy state than the other (say, it’s moving faster). So in order to compress two particles “on top” of each other, you have to apply enough force that you’re basically pushing one of the particles up to a higher energy. For large numbers of particles, this can get hard.

Different kinds of matter degeneracy happen at different pressures. At the center of a neutron star, the pressure is so high that neutrons become degenerate and start stacking up on top of each other. The matter in white dwarfs and neutron star crusts, on the other hand, exhibits only electron degeneracy. Whew! So, how bad is this going to be? Well, looking here, we find the formula for electron degeneracy pressure to be:

P = ( (pi^2*((planck’s constant)/(2*pi))^2)/(5*(mass of an electron)*(mass of a proton)^(5/3)) ) * (3/pi)^(2/3) * (density/(ratio of electrons to protons))^(5/3)

(I’m assuming that our little white-dwarf-chunk will not be so warm that the particles will be moving at relativistic speeds; if not, we have to switch to a different formula, found here.)

So, we know the density; the ratio of electrons to protons has to be 1 (Otherwise the white dwarf would have an electric charge. Of course, if you can somehow find a positively charged white dwarf somewhere, you can reduce your required pressure noticeably!); and everything else here is a constant. Plugging this in we get:

P = ( (pi^2*((6.626068 * 10^-34 m^2 kg / s)/(2*pi))^2)/(5*(9.10938188 * 10^-31 kilograms)*(1.67262158 * 10^-27 kilograms)^(5/3)) ) * (3/pi)^(2/3) * (1.00023843 * 10^9 kg/m^3)^(5/3) = 9.91935718 × 10^21 kg m^-1 s^-2

Or in other words, if we neglect the assistance that the white dwarf material will be providing in holding itself together in terms of gravitational pull, the shell for our planetoid would need to be able to withstand 9.91935718 × 10^21 Pascals of pressure in order to keep all of that degenerate matter in. That’s a bit of a problem. Actually, it’s more than a bit of a problem. It’s most likely impossible. The strongest currently known material in the entire universe is the carbon nanotube, and it has a theoretical maximum tensile strength (I think tensile strength is what we want to be looking at here) of more like 10^11 Pascals. So you’d have to find a material that could do better than that by a power of like 10^10. But, hey, that’s just an engineering problem, right?

CONCLUSION

So what the above basically tells us is this. As long as you can do one of the following things:

1. Hijack Halley’s Comet and collapse all of its mass down into a volume with diameter 5.35 milliangstroms, to create a small black hole

2. Build a pressure vessel capable of withstanding 10²² Pascals of stress, then trap inside a big chunk torn out of a white dwarf

Then you, too, can have a Super Mario Galaxy style planetoid in your very own space station! Now, given, these things aren’t easy, and possibly not even possible. But isn’t it worth it?

DISCLAIMERS

The above analysis has some limitations which should be kept in mind.

• In the case of the black hole strategy, you’d have to somehow stabilize the position of the black hole relative to the shell, so that the surface of the shell always stayed exactly 115 feet away from the black hole– otherwise random drift would cause one side or other of the shell to gradually drift toward the black hole and eventually cause the whole thing to fall in. Which probably would be kind of fun to watch, but isn’t what you want if you’re standing on it at the time.
• In the case of the solid-body/white dwarf strategy, I am assuming that the final planetoid can be treated like a point mass. This is almost certainly wrong. I’m not sure how to go about figuring out exactly the gravitational force from a chunk of matter which rather than being treated like a point is distributed through a sphere immediately underfoot.
• On the other hand, it might be interesting to find out, because if you knew how to do that you’d probably know also how to figure out the gravitational force from matter distributed through an irregular body. Most of the planets in Mario Galaxy are not spheres! There’s also planetoids shaped like barbells, and avocados, and cubes. Most interestingly, there are a handful of planetoids in certain places in Mario Galaxy shaped like toruses (doughnuts). I’m curious but not sure how to figure out, if you actually were to construct such a thing in real life, what would the gravity when walking on it be like? What would happen if you tried to walk onto the inside rim?
• None of these planetoids would be able to maintain an atmosphere. The escape velocity is just too stupidly low. (Puzzlingly, this does not seem to matter in Mario Galaxy, since Mario seems to be able to breathe even when floating out in deep space. One would be tempted to simply conclude that Mario, being a cartoon character, does not need air, but no, there are sections in the game with water and Mario suffocates if he stays underwater too long. Apparently the Great Galaxy is permeated with some kind of breathable aether?)
• Even the gravity and breathable aether aside, many the elements of Super Mario Galaxy do not seem to be possible to replicate under normal physics. For one thing no matter where Mario walks on any structure anywhere in the game, his shadow is always cast “down” underneath his feet, as if light in Mario’s universe always falls directly toward the nearest source of gravity, or if the shadows weren’t cast by light at all but were simply visual markers in a video game allowing the player to tell where they are about to land.
• I am not a licensed structural engineer, space architect or astrophysicist. The data above is provided on an “as is” basis without any representations or warranties and should not be used in the construction of any actual space vessel or celestial object. John Carmack, this means you.

Special thanks to pervect at Physicsforums for help with the degenerate matter stuff, and Treedub for pointing out what the electron/proton ratio of a white dwarf would be.

SHORT VERSION

So I’ve made this website for building blog communities, and it’s called datafall.org. Datafall lets you start these things called “blogcircles”. If there’s some group of people– maybe the people from your web forum, or your fish club, or just your circle of friends– that you know have blogs, you can go start a blogcircle for those people, and then send those people a link to it. Then all they have to do is hit the “click here to join this blogcircle” link at the top of the page and enter the URL of their blog, and from then on, whenever they post something at their blog it will appear at your circle at Datafall, too, with a link back to the blog that posted it. (Datafall doesn’t host any blogs on the website itself– if that’s what you want, there are lots of great free websites for that. What Datafall does is bring blogs together.) Finally (as you can see in my own sidebar, on the right of my own blog’s main page) once you’ve got everyone hooked up to the blogcircle, Datafall gives you several ways of embedding the stream of links from your blogcircles right into your blog, so that your own blog can show in realtime a list of the new posts by all your friends without you having to do anything.

In other words, Datafall is an RSS Aggregator, except that normal RSS Aggregators are controlled by just one person and read by just one person, and the blogcircles at Datafall are open to the world.

If you want to see how this works in practice, take a look at the Platformers community blogcircle, which is the first (and as of this writing only) blogcircle on the site; I set it up for the people I know at Platformers.net, a website that has a small gaming forum I belong to. I hope you’ll consider setting up a blogcircle for the people you know, too.

LONG VERSION

This next part mostly has to do with the history of the website at datafall.org right now, and this may or may not be of any interest to you. But, here it is anyhow.

I actually started Datafall something like a year and a half ago, but until the last couple of weeks it was actually a completely different site. The original Datafall was kind of modeled to be an RSS-based implementation of Scoop, which is the engine that DailyKos and Kuro5hin use. Scoop sites are kind of like little Slashdots, with a stream of special “official” blog posts on the front page approved in some way by either the site operators or the community of users, and then a stream of “diaries” freely posted on the side pages by normal users. My thought was that I’d used a couple of Scoop-based sites and liked them a lot; but the problem was it was hard for Scoop communities to get started, and they died really easily, since being a user on a Scoop site is kind of an all-or-nothing proposition. Becoming a user on a Scoop site effectively meant either starting a blog there or moving your existing blog, and once your blog there was set up there it was likely no one would be reading it except the other site users– so you kind of had a lot invested in the site. Scoop sites which have a real critical mass to them, like DailyKos, could make this argument easily and thrived; others couldn’t and struggled.

I looked at this situation and thought: Scoop sites has great community integration features. But they usually aren’t very good [i]blog[/i] sites. So, why not make something Scoop-like which has all of Scoop’s great community features– but which doesn’t try to be a blog site at all? The blog posts could be posted elsewhere, and the scoop site could just slurp them from RSS, and categorize and link to them. So I installed Ruby on Rails and did some tinkering, and this is basically what the first Datafall was. It looked exactly like a Scoop site, but if you clicked on any of the posts you’d find yourself on an external blog.

A year passed, and I eventually realized two things: One, I hated Ruby On Rails; and two, nobody was using my Scoop-style Datafall site, nor did it seem likely anyone was going to start doing so in future. Why would they? The site didn’t really give you any reason to use it; the site had all these community features, but these features only made sense if the site already had a big community, and this site didn’t. The only people who were using Datafall were the people who knew me from this Platformers forum I visit; Datafall was basically being used as a blog tracker for the Platformers forum users. Which was actually kind of neat, but it meant most of the Scoop-style features were useless.

So, okay then, I thought, if these Scoop-style features aren’t any use for the way people are using the site, then why keep those features? So I deleted the whole site, installed Django to replace the Ruby On Rails stuff that wasn’t working, and started over. The result is the site you see there now. The database contents, and the CSS file moved over from the old site to the new one; everything else is new.

So, what’s the point of the new site?

The idea for the new site can actually be seen in one of my “to do” bullet points for the old site. If you’ve ever used LiveJournal, you’ve probably seen these “groups” they have. LiveJournal groups are like little group blogs, where anyone with a LiveJournal account– when they write up a blog post– can choose to drop the post to the LiveJournal group instead of the normal blog. Which is really neat, but of course it has the problem that you have to have a LiveJournal account in order to use it. This doesn’t make much of a difference since you can of course start a LiveJournal account just to post in the groups, but in a certain sense this still is a little bit like the “all or nothing” problem I mention with Scoop– you can take advantage of this neat blog community feature on this one site, but unless you just happen to host your blog on the same site then a lot of the community integration is lost. I thought– as long as the point of Datafall is to offer blog community-building features, based around using RSS to paste different sites together– that replicating this groups feature on Datafall would someday make sense too. But at first I assumed that this was something to put off until the site had grown a bit– since after all, who would join these groups if the site doesn’t have any users yet?

On the other hand, Datafall in the pseudo-Scoop era was, if you think about it, basically like one little LiveJournal group unto itself– the Platformers community LiveJournal group, say. No one there wanted to use the Scoop-style features, but there was this group of people in this existing, external community who had blogs and were using the Datafall site for LiveJournal-group-style features. Looking at this I figured, well, if the people on Platformers are using Datafall for this purpose, might there also be other small net communities who might be interested in doing so too, as long as the site supported it?

So, that’s basically what Blogcircles are: Blogcircles are kind of like the “groups” on LiveJournal, but posts can go there from any blog, myspace page, any website at all, not just the blogs on LiveJournal. And hopefully this gives the reason why people would want to use Datafall in its new form: because there are people who already have some little community they’re in in which the community members just happen to have blogs, and Datafall gives some way for that community to organize itself.

OKAY, SO HOW DO I USE THIS THING?

If you go to Datafall, you’ll see a handful of links on the front page. Feel free to browse the feeds and blogcircle[s] already on the site, but probably what you want to do is either add a new RSS feed to Datafall, or add a new Blogcircle. (Don’t worry about making an account; this will be done automatically once you start doing things.)

Let’s say you want to add a new blogcircle. Hit the “Add a blogcircle” link, and fill out the form– all you really need to give it is a name, but if you want you can also add a description and a URL of your choice. (If you’re not already logged in, the form will also have you create a new account.) Once you’ve created your blogcircle, when you look at that blogcircle logged in there will be some extra options visible to you– as the owner of the blogcircle– which don’t appear for anyone else. Specifically you’ll be able to edit the blogcircle’s information, or “attach a feed”. You might actually want to do the second one of these– what this means is that you can add a special item to the Datafall sidebar, visible whenever anyone looks at the blogcircle, containing the contents of an RSS feed of your choice. (For example, remember me mentioning the blogcircle for the “Platformers” video game forum? Well, on the Platformers blogcircle, the “attached feed” shows the recent front-page posts on Platformers itself.)

Alternately, let’s say you want to add your blog to Datafall. Datafall calls the blogs it’s keeping track of “feeds” (since, after all, they might not be a blog exactly). You’ll probably want to do this in the context of adding your blog to a blogcircle– if you want, you can just add your feed now and join a blogcircle later once more blogcircles have started, but I don’t have the site to the point yet where you can get a lot of use out of it without being in some blogcircle! Maybe later. So what you’ll probably want to do for now is go to the page for a blogcircle you want to join; if you look, you’ll see at the top a link that says “Click here to join this blogcircle”. Hit that, and a form will appear asking for the URL of the website you want to add (and creating an account if you aren’t signed in already). That’s it! Your most recent post will appear on the blogcircle immediately, and when you make posts in future Datafall will notice and add those to the blogcircle, too.

Note that once you’ve logged in, the front page will appear as a summary of all the different blogcirlces you belong to; the links that are normally in the front page directory can after you’ve logged in be found in the sidebar to the right.

One last thing you might want to do, if you’ve found or started a blogcircle you really like, is embed the blogcircle into your own blog in such a way that anyone visiting your blog can see what’s been posted in your blogcircle lately without having to go all the way to Datafall. I am trying to set up Datafall so as to make it simple to embed a “faucet” from Datafall into absolutely any web page, anywhere– although how you do it may be different depending on where your site is hosted. Maybe I’m biased because I’m trying to sell you on this Datafall thing I made here, but this is actually a feature of a kind I’ve been wishing blogs had for a long time. Most blogs have a little “blogroll” bar on the right side of the page, linking an occasionally huge number of different blogs that the blogger likes; but you usually don’t have any idea which, if any, of these different blogs actually have new content. I think it would be neat if instead of forcing viewers to check each item on your blogroll manually, you could just show them an up-to-the-minute listing of all the newest posts by people on your blogroll. The Datafall embedding feature tries to be a step toward that.

If you want to try to do this, what you should do is go to Datafall and look on the right-hand sidebar, underneath where the login box normally would be. On some pages on Datafall, particularly blogcircles, there will be a little box here labeled “Embed”. This box will contain a link, which will take you to a page containing instructions on how to embed the live listing from that specific particular page somewhere else. The instructions page in question will have different instructions for different kinds of websites and blogs– blogspot accounts, WordPress blogs, plain html sites, etc– and most of the instructions will consist of a large block of HTML which you’re supposed to paste somewhere or other. Part of the reason why I have different instructions for each different kind of blog is I’m trying to provide some way of embedding that can blend into your site completely seamlessly– I’m trying to set things up so that if you embed a blogcircle in a webpage it looks like it was designed to be there. Note, though, that although , as, I only have a few things listed there now. If you have a blog or website that isn’t covered by the instructions on that page, then please do post in the comments below, tell me what kind of blog it is and why it is that the existing instructions don’t work, and I’ll see if I can add a section for your blog type.

IS THAT IT?

So this is basically what Datafall is right now. I’m still actively working on it, and since the new site is a lot easier to make changes on than the old one I hopefully should be able to do them at a potentially fast clip. I’m happy to take any suggestions for improvements, and I’ve got a list of improvements I’m going to try to add as soon as I can. Here are some of the things I want to work on with Datafall in the future:

• Right now you can’t post to any blogcircle except one you’ve specifically joined– and once you’ve joined, you can’t not post to it. Every post you make on the blog will appear on all of your blogcircles, period, and you can’t remove them. This needs to be fixed stat. You should be able to add yourself to a blogcircle “conditionally”, such that your posts are displayed on the blogcircle only when you assign them to be rather than automatically; you should be able to withdraw a post from a blogcircle or from datafall if you want; and ultimately I think it would be neat if there were “open” blogcircles that anyone could post to, whether they’ve joined the blogcircle or not. (So for example there maybe be like a “Science” blogcircle, and any individual post from any feed on Datafall could be assigned to appear on the Science blogcircle so long as it had science content.) This kind of hands-on way of using Datafall probably isn’t the way most people would want to use it– better to just use it the normal way and have the site do everything for you automatically– but it should at least be an option.
• Right now there really aren’t any limits on who can join what blogcircle. This could potentially be kind of bad; in many cases it would make sense for some kinds of blogcircles to be able to control their membership and content. There needs to be the ability for the operator of a blogcircle to remove feeds and posts that aren’t appropriate to that blogcircle, and it needs to be possible to set a blogcircle such that when people click “join blogcircle” they aren’t instantly added, but have to be approved first. (Conversely, if there’s a feed on Datafall you like or think is appropriate to a particular blogcircle, maybe it should be impossible to invite people.)
• Right now the only person who can do any kind of maintenance on a blogcircle is the person who created it. The blogcircle owner should be able to delegate authority. This doesn’t make much difference right now, when there’s very little that even the blogcircle owner is able to do, but once the blogcircle owner gains the ability to delete posts approve feeds etc the blogcircle owner should be able to also give select members of the blogcircle the ability to do same.
• Combining the above three ideas together,
• AJAX. If you don’t know what AJAX is, then don’t worry about it too much, but in my book this is a biggie. The old Datafall had some great AJAXy features– this was the one thing Ruby On Rails was good at– but the new Datafall has none, mostly because. Incidentally, if anyone can recommend a Python library for AJAX generation hopefully analagous to RJS for Ruby, please let me know.

Okay, that’s it.

Computer Science can be said to predate even the idea of a computer itself by at least two millennia, in the sense that even before Babbage started designing calculating engines (for simplicity, I will here ignore mechanical calculation devices which predate Babbage), mathematicians had for a long time been analyzing “algorithms”, mathematical procedures which behave like functions and are calculated by following a specific set of completely prescribed steps. In fact, although the name was not given to them until the 17th or 18th century, algorithms can be said to go at least as far back as Euclid in 300 BC. Since algorithms, which are the fundamental basis of computer programming, are basically just computer programs without the syntax, it at first seems surprising that they would predate machines capable of running them by so much; but this really isn’t so unusual, since algorithms in principle can be worked by a human with a sheet of paper– all that is necessary for all the properties of an algorithm to hold universally is that something, mechanical or human, follows the steps without room for inserting will of one’s own.

Work on understanding algorithms before the 20th century, however, was stymied by a lack of rigor in what we understood an algorithm to be. Though it did occur to mathematicians hundreds of years before Babbage that algorithms are not just something to design, but objects in their own right whose properties can be meaningfully analyzed, all such analysis ultimately was doomed by the intuitive (rather than rigorous) nature of what an algorithm was understood at the time to be. For example, I say above that an algorithm consists of a series of “steps”. But what, exactly, is a “step” in an algorithm? For a machine, such as a computer, a “step” can be easily understood to be a machine instruction. Instructions for humans, on the other hand, do not come in discrete units– if step one in an algorithm is “write down a number”, could we actually call this a “step” or should we break things down further into “pick up a pencil”, “pick a number”, “position hand over paper” etc? This sounds nitpicky, but it’s crucially important, since without knowing what a step is it is of course impossible to ask any of the questions we would like to ask about algorithms which concern steps– questions like “if this algorithm is executed, how many steps will it take?”. What was not understood in early attempts to understand algorithms was that algorithms could not really be meaningfully studied in the absence of some specific idea of a “model of computation”– a formal system by which algorithms can be specified according to some exact scheme and then deterministically executed.

The path that lead to this realization was arguably first set off by David Hilbert, who for the tenth of his famous 23 problems of 1900 asked:

The Entscheidungsproblem [decision problem for first-order logic] is solved when we know a procedure that allows for any given logical expression to decide by finitely many operations its validity or satisfiability … The Entscheidungsproblem must be considered the main problem of mathematical logic.

This question was eventually answered in 1935, before the first computer had ever been devised, in a famous paper written by one Alan Turing; in this paper the “Turing machines” that now bear his name were first defined. The answer was not the one Hilbert would have hoped for. Turing defined a simple, abstract machine which in principle could be built, but in practice would be impractical; the point of this machine was simply to provide a sort of least common denominator for computing devices. Studying this machine provided a number of surprises .The surprise which would have disappointed Hilbert was that his question from 1900 was impossible to answer: there are logical expressions whose validity or satisfiability cannot be finitely decided.

The surprise which would be more interesting to us however was the idea that the machine turns out to be universal. The Turing machine, despite its simplicity, has all the capabilities of any finite-time deterministic machine which can be built or even imagined. A related surprise was that any machine capable of emulating a Turing machine is, itself, just as powerful as a Turing machine. This gives rise to the idea of a “universal model of computation”. There are a countless number of these, and the Turing machine was simply the first to be known. Any such universal model provides the rigor needed to analyze algorithms; whichever one you choose, or whichever one you mechanically implement, things will behave in certain important ways the same.

Interestingly, however, though we tend to think of it as sort of the platonic form of a computational model, the Turing machine was not the first universal model of computation ever invented. The first was in fact invented without anyone having fully realized its significance, by Alan Turing’s teacher, Alonzo Church. Church had years before Turing’s paper defined a “Lambda calculus”, a sort of way of defining trees of functions that call other functions, equipped with a rule for simplifying these trees as if the function’s value were being evaluated. (The function’s value turns out to be another tree of functions that call other functions.) The lambda calculus is if anything even more simple than the Turing machine, giving it valid claim to be more fundamental than the Turing machine; and it has a certain strange beauty, in that “programs” written in it build all normal accouterments of programming out of nothing but functions. (Numbers, for example, are encoded by chaining functions together in a way that they “count” themselves off when evaluated.) Unfortunately the lambda calculus is completely alien to anything encountered in human experience, and is infamously difficult to understand; so difficult, in fact, that the fact the Lambda calculus is just as powerful as the Turing machine was not realized until the 1950s, decades after Turing’s paper, and the proof was written by someone other than Alonzo Church. Because it failed to gain historical precedence and because it is neither as easy to grasp nor as close in resemblance to a normal computer as the Turing machine, the Lambda calculus is today largely unknown, although in an odd twist a rephrasing of the Lambda calculus (the calculus of logical combinators) is still used today deep in the guts of the virtual machines of certain so-called “functional programming” languages, as a sort of abstract “machine code” the virtual machine evaluates.

The Lambda calculus looks like this:

((λ (a b c d e f g h i j k l) ((λ (m n o p q r) ((λ (s t u) ((λ (v w x) (λ (y) (λ (z) (i (v z) z (n z (f c) (λ (aa ab) (j (k aa) ab)) l (t q (x (w y z) (s q (((f c) l) z))))))))) (λ (v) (k ((o l) (s b ((o (e l)) v))))) (λ (v w) (u (g p e) (s e v) (s e ((c (c (d (f l)))) w)))) ((λ (v w) (λ (x y) (u (g p q) (w y) (v x)))) ((λ (v w x y z) (λ (aa) (z (g p q) (y (g p q) (x v p aa)) (x w d (((c o) l) aa))))) (λ (v) (r (((c l) v) a) ((((o c) l) v) a))) (λ (v) ((λ (w) (r (w a) ((λ (x) (r (x a) (r ((x b) a) (((o l) x) a)))) ((p l) w)))) ((o (d l)) v))) ((λ (v w) (λ (x y z) (m (g q p) ((q (p (w x))) (v y z)) b))) (λ (v w) (m v (m v w b) (j b ((v l) w)))) (λ (v) (λ (w) (j (v w) w)))) (λ (v w) (n w v (λ (x y) (j ((k x) a) y)) l b)) ((λ (v) (λ (w x y) (n (j a (j x y)) w (λ (z aa) (j (((k z) k) (((k (k (l z))) k) (((k (l (l z))) k) a))) aa)) (λ (z) (j ((i (k z) g v) (k (k (l z))) (k (l (l z)))) (j (l (k (l z))) (l (l (l z)))))) b))) (λ (v w) ((v a) w)))) ((λ (v w) (λ (x) (v q (w q (v q x))))) (λ (v w) (n w v (λ (x x) ((λ (y z aa) (y aa aa x (y aa z (l (l (l (l x)))) x))) (λ (y z aa aa) (j (k aa) (j (k (l aa)) (j (r (k (l (l aa))) (y (k aa) (k (l aa)))) (j (r (k (l (l (l aa)))) (z (k aa) (k (l aa)))) aa))))) (λ (y z) (((y a) z) a)) (λ (y z) (y (z a))))) (p l) b)) (λ (v w) (n w v (λ (x x) ((λ (y) (j ((((x b) b) b) a) (j (((y b) b) a) (j ((y b) a) (j ((x b) a) (j (((x b) b) a) (j ((((y b) b) b) a) (j (x a) (j (y a) x))))))))) ((((x b) b) b) b))) (p l) b)))))) ((λ (s) (λ (t u) (n u t (λ (v w) (s (k v) w)) l b))) ((λ (s) (λ (t u) (n (s t) p (λ (v w) (j (k v) w)) (λ (v) (s (l v))) u))) (λ (s) ((s (λ (t) (j ((t a) a) (h (t b) (t a))))) (λ (t) a))))) ((λ (s) (λ (t u) (s t (c o) (s (g c t) o (s (g o t) c u))))) (λ (s t u) (n u s (λ (v w) (j (h (k v) (g t (k (l v)))) w)) (c l) b))) (λ (s t u) (n (j t u) s (λ (v w) (j (r (k (k v)) (k (l v))) w)) (λ (v) (j (l (k v)) (l (l v)))) b)))) (λ (m n o) ((λ (p) (((m p) (j n o)) b)) (λ (p) (j ((p a) b) (j ((p a) a) (p b)))))) (λ (m n o p q) ((k ((n (λ (r) (j (λ (s) ((k r) (o (l r) s))) (p (l r))))) (j (λ (r) r) m))) q)) (c c) (d c) (g (f c) (g d e)) (λ (m n) ((m n) ((n m) a))))) (λ (a) (λ (b) b)) (λ (a) a) (λ (a) (λ (b) (a (a b)))) (λ (a) (λ (b) (a (a (a b))))) (λ (a) (λ (b) (a (a (a (a (a b))))))) (λ (a) (λ (b) (a (a (a (a (a (a (a b))))))))) (λ (a b) (λ (c) (a (b c)))) (λ (a b) (λ (c) (λ (d) ((b c) ((a c) d))))) (λ (a b c) ((a (λ (d) c)) b)) (λ (a b) (λ (c) ((c (λ (d) b)) a))) (λ (a) (a (λ (b) (λ (c) c)))) (λ (a) (a (λ (b) b))))

This is a copy of a letter I wrote to various congressthings. I reproduce it here because I didn’t write a blog post this weekend, and because you probably won’t be able to tell the difference between this and a blog post anyway. Enjoy.

* * * * *

Dear [Whoever],

I am writing about the recent effective dismantling of NASA’s Beyond Einstein program, and in general the massive slashing of science research within NASA’s budget in the last few years. This is an issue which has received almost no popular attention, and which I suspect Congress has not had the chance to seriously consider. However, I feel that preserving these science programs, and Beyond Einstein in specific, is in the long term of great importance to America.

Beyond Einstein¹ is an umbrella program by which NASA is performing a series of flight experiments to gather information about four cosmological phenomena at the limit of human understanding: black holes, gravitational waves, dark energy, and cosmic inflation. All four of these things are an accepted part of modern science, but their exact workings are very poorly understood. Data about their operation would be invaluable to physicists, who must develop theories to explain these things even though they are astronomical phenomena and cannot be observed in a lab.

Unfortunately, though, NASA has in the last few years been de-emphasizing science in its funding, and one of the many worthy programs that may not survive this shift in priorities is Beyond Einstein. The following is a quote from Steinn Sigurðsson, a physicist who provides a stark account² of an NRC meeting where the future direction of the Beyond Einstein missions was discussed:

At the request of the DoE, the NRC is doing a priority ranking, a funding wedge is opening in 2009, one mission can get a startup, the others, not so much.

Realistically, a second mission will probably be named as a ranked priority, then the rest will get bounced to the decadal survey that will start in a couple of years, and we start all over again. If there is any funding for new starts again (we’re looking at maybe 2022-2025 at current budget profiles).
…
No one is going to win this, only lose. It should never have come to this.

The stakes are high; literally thousands of scientists are looking at the core science activity they have chosen to work in being annihilated for 10-20 years, a lot of junior people could be dumped from science, a lot of senior people could look at having the field they worked to build being shut down.

It is an indescribabl[e] waste, for what is a surprisingly small amount of money on the scale of the US economy. The funding gap that is squeezing the Beyond Einstein Program out of existence is about 2-300 million dollars per year - that, over ~ 15 years is what it would take to do 3-5 of the missions in quick overlapping succession.

I would like to call attention to something that may not be obvious in this quote. Because physics experiments, like a NASA probe or a particle collider, are such concrete things, it is easy to lose sight of exactly how much accumulated effort goes into them. Cancelling or going forward with an experiment like this seems like a simple decision: either you build it and you have one, or you don’t. It is easy to forget the fact that the experiment is not just a manufactured physical object, but an entire section of the scientific community who are working on the experiment and dependent on it going forward. Besides just the people designing the probe, just one of these experiments inevitably leads to years of papers and advancements just analyzing the collected results; canceling the experiment means shutting all of that down.

Meanwhile, going forward with just one of the projects is not much of a compromise, since Beyond Einstein is a survey, not a single experiment; some of the Beyond Einstein experiments are on drastically different subjects, and so deciding to perform only one of the experiments means telling physical science it will be given the opportunity to move forward on some subjects, but not others.

It is easy to shrug off the sciences, especially edge science like Beyond Einstein represents, as optional or inessential. However foundational scientific research like this is, in the long term, what drives real scientific and technological progress. Since Beyond Einstein covers the least well-understood aspects of physics, it has the real opportunity to spark serious advancements in scientific theory. By limiting its scope, we risk missing that opportunity.

It is my hope that when the new budget comes up for consideration over the next few months, the Congress will act to ensure the Beyond Einstein program receives adequate funding to complete its mission. The current NASA budget request as I understand it does not serve this goal, instead choosing to focus its funding efforts largely on new manned spaceflight programs. While expanding manned space exploration is a worthy goal for NASA, this goal should not be pursued at the expense of NASA’s ability to do science.

Thank you for your efforts,

[mcc]

1. http://scienceandreason.blogspot.com/2006/11/beyond-einstein.html
2. http://scienceblogs.com/catdynamics/2007/04/beyond_einstein_iv_showdown_in.php