stevebob.net

Reinventing the 3D Wheel

This is an explanation of the 3D drawing techniques that have been used on this site. During the life of this site I have implemented and re-implemented linear algebra libraries and 3D demos which used them.

Here are some demos that make use of 3D:

Doing everything from scratch

When I started writing demos for this site I had an aversion to using any third party libraries. I wanted to learn how everything worked and that meant shunning existing technology in favour of writing everything myself from scratch. Many of the demos on this site are 3D but none make use of WebGL, three.js or any other well established 3D web technology. All the demos use a 2D HTML canvas context, with all the maths to make images appear 3D implemented in javascript.

High School Technical Drawing

My first attempt at a 3D engine was based very closely on a technique I learnt in my high school graphics class. This class focussed on technical drawing both by hand on paper and using CAD software. We learnt how to do perspective projection on paper in the following way:

Perspective projection I learnt in high school

Here’s how to hand draw a wireframe cube using perspective projection as per the diagram above:

Start by drawing lines from each point of the top view to the eye.

From the point where each of these lines crosses the horizon, draw a line straightdown until it intersects the picture plane.

From each point in the top view, draw a line straight down until it intersects thepicture plane, then begin a new line from that point to the vanishing point.

For each point in the top view with a height (position on the up/down axis) of 0,it's position in the perspective projection is the intersection of its correspondinggreen and blue lines.

For each point in the front view (besides the ones with 0 height), draw a horizontal line through that point which intersects all green lines. From each intersection of a line from a point on the front view with a green line from a corresponding point in the top view,draw a line from the intersection point to the vanishing point.

The intersections of the cyan lines withthe corresponding blue lines are the positionsof the corresponding points in the perspective projection.

Now that the points are all present, draw lines between the ones between which edges exist in the shape you’re trying to draw.

Automating the process

As I knew how to draw perspective projections of shapes by hand, the next step was to write a program to draw shapes using the same technique. I actually wrote a QBasic program to do this for flat objects (where all points have 0 height) while in my “experimenting with MS-DOS” phase in high school, and then re-implemented it in javascript for non-flat objects based on the same maths a year or so later.

For a computer to be able to draw perspective projections, it needs a way to compute the 2D coordinates for drawing on the screen, from a given point specified using 3D coordinates. Repeatedly doing this for each point of a 3D object, and drawing lines between the corresponding points on the screen, results in its perspective projection.

Basic idea of similar triangles

So how to convert a 3D point into a 2D point which can be drawn on a screen? My solution used Similar Triangles As shown in the diagram on the left, if you have 2 right-angle triangles with the same ratio of side lengths, and you know the side lengths of one of them, and the length of one of the sides of the other, the length of the remaining side can be derived.

The perspective projection above is full of triangles, so all I needed to do was find some triangles to help compute the 2D point on the perspective projection from the information known about the point.

Below is the triangle used for finding the horizontal point (called ‘a’ in the diagram).

Method for finding the horizontal position of a 2D point in a perspective projection

Once the horizontal position of the point is known, it can be used to compute the vertical position (referred to here as ‘b’) in a similar way:

Method for finding the vertical position of a 2D point in a perspective projection

Once this formula for converting between 3D and 2D points is implemented, it’s some simple bookkeeping to know which 2D points must be connected to complete the wireframe image. This technique proved suitable for several applications where it wasn’t necessary to ever draw solid polygons or other non-wireframe shapes. Here’s some demos that use it:

The “Proper” Way

In 2012 I took UNSW’s Computer Graphics course with Tim Lambert. Each week I would learn something more about how conventional 3D engines work, and I thought it would be neat to build my own 3D engine (entirely in javascript with no video hardware support) and add features as I learnt about them. Here’s how that engine evolved as the course progressed.

This section has several interactive demos of the engine at various stages of completion. To control the eye’s point of view in the demos, use WASD keys to move forwards, left, backwards and right respectively, and the up, down, left and right arrow keys to move up, down, rotate left and rotate right respectively.

Perspective Transform

Rather than using a complicated formula to compute the position on screen of a 3D point using perspective projection, one can use matrix multiplication to achieve the same effect. Represent the point (x, y, z) as a vector in R4 (x, y, z, 0) and multiply it by the matrix:

Perspective transform matrix

This results in a vector in R4 where the first 2 values are the x and y coordinates on the screen to draw that point. The ‘d’ in that matrix refers to the “depth of field”, or “how rapidly points vanish towards the centre of vision”. This image was rendered using a high depth:

High depth example

This was rendered using a low depth:

Low depth example

Here’s a demo that does a perspective transform and nothing else:

Wireframe

Back-face Culling

For this to make sense we need to add faces to our 3D engine. So far all we have been able to draw were points and lines. Now we need to add solid polygons. Think of this as taking a 3D model made from wire and attaching paper cutouts to it. In the real world paper cutouts have a front and a back and are obviously visible from both sides. When our wireframe is completely covered by cutouts, each cutout will have one side (the inside side) which cannot be seen from outside the object.

Having the faces be visible from both sides means more work for the 3D engine. As faces are flat, only one side of the face can be seen at any point, so if we only draw the visible side we generally halve the number of faces we need to consider at any point. This is the idea behind back face culling. Each face is given a front and back face, where the front face faces the outside of the object. With back-face culling, the backs of faces are not drawn.

Here’s a demo of a cube drawn using back-face culling:

Backface Culling

Note that in the simple cube example above it appears that faces are drawn in the “correct” order, in that things in the foreground obscure the things behind them. Back-face culling is sufficient for correct draw order in convex shapes such as cubes, though is insufficient for more complex examples:

Backface Culling 2 Cubes

Clipping Plane

Using the similar triangles formula or the matrix multiplication, points behind the eye still get assigned points on the screen. In order to not draw points which are behind the eye, we need a way to remove all these points before rendering. Using a clipping plane to “slice” through the 3D model of the world before rendering is one way to do this. This example places the clipping plane in front of the eye so its effect is more visible. Back-face culling is also in use which can be seen when part of the cube is clipped and the inside is not visible.

Clipping Plane

Binary Space Partitioning

A 3D model is generally represented as a list of polygons, each representing a face. To correctly render the model, the faces must be drawn in a specific order so that things in the foreground cover up the things behind them. Thus we need a way to sort faces into a suitable order such that if they are drawn in that order it looks “correct”. This is not as simple as it may seem, as there may exist mutually overlapping faces, such as the bottom of a cardboard box, where every face is obscured by some other face. Thus we can’t just sort the faces as no face could be drawn last and thus there there may be no order in which we can draw the faces.

The solution is to spit some faces so as to remove all mutually overlapping faces, and sort these faces in the correct order. One algorithm which achieves this is “Binary Space Partitioning”. This involves creating a binary tree of faces known as a Binary Space Partition Tree (BSP Tree), where each face has a front and back side. For each face in this tree, all faces in the right subtree are completely in front of the current face, and all faces in the left subtree are completely behind it. For a face A to be completely in front of or behind some other face B, it means that the plane which contains face B does not intersect face A.

Once this tree is created, faces can be drawn in order by traversing the tree in-order (left subtree, then root, then right subtree). The algorithm for inserting a face into a BSP Tree is as follows:

bsp_insert(face, tree):    if (tree.value == null):        tree.value = face    else if (face is entirely in front of tree.value):        bsp_insert(face, tree.right)    else if (face is entirely behind tree.value):        bsp_insert(face, tree.left)    else:        face_front = part of face in front of tree.value        face_behind = part of face behind tree.value        bsp_insert(face_front, tree.right)        bsp_insert(face_behind, tree.left)

Depending on the order in which faces are added, the number of split faces may vary. It is ideal to minimize the number of splits and thus to reduce the eventual number of faces making it faster to draw them. Generally the list is shuffled and repeatedly inserted into new trees until a reasonably small tree is found. A BSP Tree need only be generated once ever for a given 3D environment, so for games which use BSP Trees, one is pre-compiled when the game is made and the game comes with all its BSP Trees already computed.

A downside of BSP Trees is that they must be regenerated when anything changes in the environment. This makes them not ideal for rendering moveable characters in games.

Here’s a demo showing how a particular 3D environment is split up by the BSP algorithm.

BSP Wireframe

Here’s the same demo with solid faces:

BSP Demo

And here’s another demo using the same technology to render rooms:

Rooms

Conclusion

I don’t intend to actually ever use this 3D engine for anything practical. In reality, massively parallel video hardware is used to implement Z-Buffering - a technique which can be used in place of BSP, and libraries exist to remove the need to implement all these low level features. These demos really just served as a learning exercise to myself. Reinventing the wheel is a great way to understand how wheels work.

Heroes and Villains

This is an explanation of a graphical demo I call “Heroes and Villains”. I wrote the first version in my first year of university and it was one of the first demos on this site. I revisited the code several years later when I entered the demo in JS1K competition, and re-implemented it in less than 1 kilobyte.

Demo runs for 5 seconds then pauses (so as to not drain your laptop's battery).
Click the demo to resume. Subsequent clicks restart the demo.

Etymology

The demo is named after a “getting to know you” game. Take a room full of people and tell each of them to choose one person to be their “hero”, and choose a second person to be their “villain”, keeping the identity of both a secret. Then tell each person to move such that their hero is between themselves, and their villain. As each person moves into position, they force other people to keep moving, as the position of each person affects the position of people who have chosen them as heroes or villains. Played indoors in groups of around 15, people tend to stand in lines moving sideways or radially about the centre of the room, and tend to also move outwards getting as close to walls as possible.

Naturally, the question arises: “Will the system ever reach a stable state, where everyone’s hero is between themselves and their villain?” This demo started out as an attempt to simulate the game with a large number of players in a large space over a long period.

How it works

Initialization

Each player's hero's villain is that player. Likewise for each player's villain.

Initially, each player is randomly assigned a position, and a pair of other players - one its hero, and the other its villain. When a player A is assigned as the hero of some other player B, B gets assigned as the villain of A. The likewise is true when a player is assigned as a villain. See the diagram to the right for a graphical explanation.

This reflexive property is not part of the original game, and several hero/villain assignment policies were experimented with, and this one seemed the most interesting to watch.

Simulation

The simulation is broken into discrete ticks. That is, there is an operation that is repeated infinitely at some frequency. This operation consists of calculating the next position of each player based on their current positions, updating the position of each player, and drawing each player to the display.

The rules of the simulation used in the calculation of each player’s position are as follows.

If a player’s hero is between themselves and their villain, the player stays where it is.
Otherwise, a player moves towards the nearest point that would put its hero between itself and its villain.

In order to calculate the point a player will move to, there are two scenarios depending on the relative positions of the player, their hero and their villain. Scenario 1 is assumed, and the point that would be used is checked to see if it places the hero between the player and its villain. If this property is not met, scenario 2 is used instead. Once the target is found, the player moves some small distance towards it.

Scenario 1

Move perpendicular to the line between the hero and villain.

Scenario 2

Get "just behind" the hero. The distance between the hero and the target is some small constant.

Variants

Since implementing the original version, I have created some variants:

Original
3D
4D(with colour representing the position on the new axis)
JS1K

Don't Plan Your Trip

My name in the book at the summit of Tromsdalstinden

I’m not a planner. To be precise, I have a high threshold for plan-worthy-ness of activities. Well below this threshold is traveling. Traveling through Europe with no plan or itinerary is more expensive and stressful than the alternative, as transportation and accommodation is generally booked at the last minute. That said, my style of traveling allows a level of freedom, that I feel justifies said expense. This is the story of my time in Norway.

Amsterdam

The story begins in Amsterdam. Robocup was finished, and I was spending a few days in Amsterdam as it’s close to Eindhoven (where Robocup was held) and I sure didn’t have anything else planned. While I was in Amsterdam, I figured out where I would go next. I decided on Oslo, for no particular reason. Norway seemed like an interesting place to visit. I booked my flight (but just the flight from Amsterdam to Oslo) and a few nights in a hostel (it had good reviews).

A canal in Amsterdam

I was waiting in the check-in line at the airport in Amsterdam. It was a long line and I was bored. Three girls lined up behind me. Let’s have a conversation. What do I say? I think they’re Norwegian - I mean they have blonde hair. Ok let’s go with that. “Er…Hi are you going back to Norway?”. Smooth.

Meeting these people turned out to be very fortunate. They gave me lots of advice about being a tourist in Norway. Most immediately useful was the fact that alcohol (and to a lesser degree, everything else) is very expensive in Norway, so stock up in duty free at the airport.

Oslo

A small house I found in the river running through Oslo

When I got to Oslo I realized I had overlooked a rather important detail. The airport is a reasonable distance from the city, it was already after midnight, and I didn’t really have a plan to get to my hostel. My new friends offered me a lift - their roommate was picking them up, and had a free spot in his car. As we drove to my hostel, one of the girls told me that she would be away from her apartment for the weekend, and I was welcome to stay there until she got back. I was also told that my hostel, despite the good reviews, was in a poor location - far away from anything interesting.

I stayed a night in my hostel, before taking up the offer of using my new friend’s apartment for the weekend. I spent the weekend getting to know the social circle of the girls I met at the airport, being shown a small but beautiful city, and finding out about life in Norway. With all this new information, I was able to properly plan my itinerary, at least for the coming week.

Bergen

I took a train from Oslo to Bergen. This was about an 8 hour ride over frost-covered mountains, even in summer. It was scenic. Here’s a photo:

I made the mistake of assuming there would be reliable internet on the train, which I could use to book a hostel and flight out of Bergen. I planned to go to Tromsø the following day. So naturally, the first thing I did when I got to Bergen was find a place with wifi and figure out where I would sleep that night. Luckily I found a place with a vacancy not far from the train station (Bergen is a pretty small place). That evening I went for a walk around the city.

Tromsø

Pronounce the “ø” like “ur” (but not like “urrr” or you sound like a noob). People familiar with the stories from when I went to Alaska will be noticing some similarities at this point (if not much, much earlier). So, at some point in my trip I had to make it my objective to go as far north as possible. My new high score for degrees of latitude is Tromsø a city in northern Norway located on a small island.

The evening I got there, I went for a walk around the island. I remember it being late when I first arrived, found an internet cafe, found a hotel and checked in. By evening, it was probably more like 9pm. I don’t remember, and it didn’t matter, as the sun was still up. When I was in Alaska the sun didn’t set until midnight, but here it didn’t set at all. But back to my walk.

I didn’t have a plan. I didn’t have a map, or a phone. No matter - the island is tiny, so I just walked, aimlessly. I ended up getting lost but someone pointed me vaguely in the right direction and that was enough to get me home. A large amount of the island, is forest, so I had lots to explore. I emerged from the forest to be met by a harrowing view. The land ended and then there was a giant stretch of water, and in the distance there were frost-capped mountains disappearing into the water. I struggle to articulate exactly why I’m scared of fjords. Something about scale. It’s the same feeling as when you imagine really deep water.

I don’t have photos (no phone, remember) but here’s one I took later of the island and the mountains in the background. At the time I was on the far side of the island.

On my last day in Tromsø I climbed a mountain named Tromsdalstinden. The windchill at the top was so bad that my mouth went numb and I was unable to speak clearly. There was a large stone structure at the top, that served as a windbreak. I met some people up there as well who fortunately could understand me despite my accent and impaired speech caused by the cold (they’re the ones who took my photo). There was also a book which I signed. I left a 10nkr piece hidden somewhere in the pile of rocks.

Oslo Again

Before leaving Norway I went to Oslo for one more night to attend the birthday party of one of the girls from the airport. Norway was the highlight of my trip to Europe. I always felt welcome and at home during my week there, and I owe that to the amazing people I had the pleasure of meeting.

Confusing Train Adventure

One hour. That’s how long you’re told it will take to start to feel the effects of the innocuous looking cake. You’ll shortly be boarding a train from Amsterdam to Eindhoven where you and some friends have a reservation for the night. There’s track work so the trip will comprise of a train ride, a bus ride, a second train ride, and finally a bus trip from the station to the hotel.

You came into this with some premonitions, and so are wary of the placebo effect, and also of the fact that your awareness of the placebo effect may cause you to ignore something legitimate. You board the first train of the journey and take a seat. It’s been a big day of traveling. You realise you’re exhausted, so you close your eyes for a bit. It’s been half an hour.

When you wake, the train is still moving. Nothing yet, though you aren’t entirely sure what to expect. Reading from the paperback you brought with you, you find yourself getting to the end of a paragraph and not knowing what you just read. That’s fine - you’re just tired - that always happens when you’re tired.

The first transfer goes without a hitch. It’s a bit hard to concentrate, but once again, you’re exhausted. You enjoy the scenery as you look out the window of the bus. It’s summer, so the sun doesn’t set until around 10pm. You think back on your premonitions. Pop culture would have you believe that you should be taking great interest in relatively trivial things. If you were a character in a television show, you would be staring out the window, transfixed on whatever was out there, ignoring all other stimuli. At this point you realise that your mouth is hanging slightly ajar, and you’ve been staring out the window for a very long time with little regard for anything else going on around you.

This happens a number of times. Each time, you take amusement in the fact that this is what society expects of you. You find yourself thinking a lot about what the people around you are thinking about you. Do they know? Are they judging you? You take out your paperback and try to get some more reading done. You read a sentence. The following sentence has some pronouns, that probably refer back to the previous sentence, but you can’t figure out which people or objects are being referred to by words like “he”, “she”, and “it”. You re-read the two sentences several times and eventually make some progress but it’s slow going, and once again, frustrated, you give up and go to sleep.

The next transfer is more difficult. You find yourself in a snack shop, staring at a shelf of food. What do you want? How do you normally deal with these situations, where the choice is essentially arbitrary. It doesn’t matter what you buy. You don’t even have to buy anything. Hurry up - you’ve been staring at this shelf for a while now. You pick out a waffle, and proceed to the counter. When asked if you’d like a drink, you nervously move towards the fridge, before being reassured that you don’t have to buy a drink, at which point you return to the counter without a drink.

On the train, you reflect on how you navigated the snack shop. You realise that the cashier probably noticed something, and that out-of-towners must put her through something similar quite frequently. “Bloody tourists!”, you imagine her thinking. This is an amusing thought, worthy in fact of sharing with your friends. You try, and it seemed to go alright - they got the gist of what you said, but think you meant that the cashier actually mentioned something to you (she didn’t). You try to clarify your point, but a few words in you pause for thought and struggle to recall how you got to that point of the sentence. Eventually you get your point across.

But your inability just then to form a coherent sentence got you thinking about what exactly is going on. You try to communicate this, but your friends just laugh at you. They say something, you reflect on what they said, realised that they asked you a question but you’ve forgotten exactly what it was. You respond as best you can.

You find this whole ordeal very amusing, and chuckle quietly to yourself. Then you realise that people are watching you laugh for apparently no reason. From their point of view you are conforming to a stereotype, and this thought is even more hilarious than the last. So you laugh harder. After several minutes of this positive feedback loop perpetuating itself, you become embarrassed and force yourself to stop.

Your latest theory is that there’s been a delay introduced between perceiving something, and processing it. You hear a sentence, it sits in a buffer for longer than usual, slowly fading from your memory, before you attempt to determine its meaning. Interesting thought, better tell everyone about it. But news of your latest epiphany is met only with laughter.

Riding the bus on the last leg of your journey, you notice something new. The sound of the engine is, at a certain frequency, repeating short bursts of a high pitched whine. You must have ridden on a hundred buses in your life and have never noticed it before. If you stop concentrating on the noise, you stop hearing it, but it resumes once you focus your attention on the sound of the engine, which you do very often, as you find it fascinating. By this point, you have enough sense to not mention it to anyone.

You arrive at your hotel, and exhausted, collapse onto your bed. You realise that the paperback you were reading is no longer with you. Where did you leave it? You think back on the events of the past few hours, and you can’t be sure at which point it left your side, but you have a suspicion that the cashier at the snack shop will be enjoying a new copy of A Storm of Swords.

Thinking Fourth Dimensionally

Great Scott!

It’s likely that everyone has encountered a work of fiction involving an extra dimension at some point. Be it time travel or trans dimensional beings, “extra dimensions” is a very popular topic. But what does it actually mean?

The Basics

If I was asked the question “What is a dimension?” I would say “A place for a number to go in a coordinate”. That is, if I was to give a coordinate of a point in a particular space, for each dimension that space has, I would need to specify one number. In a one dimensional space (think an infinitely long line), to specify any point, I need only one number, and that will tell you how far along the line my point is. In a two dimensional space (Super Mario), I can move (say) up/down and left/right, so I need to specify two numbers. Similarly, to specify a point in three dimensional space, I need three numbers. Imagine a space where you would need four numbers to specify a point. It’s very easy to imagine spaces with up to three dimensions, but due to the nature of our universe, we are not accustomed to go any higher than that.

What about time?

Time can be considered a dimension, and it’s possibly the easiest way to imagine a fourth dimension. To specify a point in the universe at a particular time, we need three numbers to specify the point (as it is a 3D space) and a fourth number to specify the time. Thinking of time as a dimension is a convenient way to help understand a fourth geometric dimension, by thinking of a three dimensional space inside the four dimensional space changing as the fourth axis is traversed. That last sentence was a bit intense, but should become clearer to you in the course of, well, time.

Analogy Time!

Imagine a pen, with an infinitely small point. If you draw an infinitely small dot on a page, we could say that the dot has zero dimensions. Now, with the pen, draw a straight horizontal line on the page. The line has one dimension. Now imagine grabbing the line and pulling it down the page, forming a square. Now take the square, and pull it up, out of the page, and you get a cube. Now what? We can go no further in a three dimensional space. So what does the cube become?

Imagine living in a two dimensional world. Suppose we inhabited a large sheet of flat paper. Three dimensional beings might take a cube, and push it through our world. Initially, there would be just some empty space. Then, depending on how they push the cube, we might suddenly see a square appear, or a rectangle, or a triangle. We would continue to see a two dimensional cross section of the cube until it completely passed through our world and vanished. If four dimensional beings want to push a four dimensional shape through our three dimensional universe, we would only be able to see a single three dimensional cross section at a time.

If you get a whole bunch of cut-out square pieces of paper and stack them together, you can make a cube. You are creating a 3D shape by stacking 2D shapes in a third dimension. Now say you had a whole bunch of cubes. You can’t stack these like you did the squares to enter a new dimension, but you can “stack” them through a fourth dimension. At any point, you can only see one of them. Think of our 3D universe as a cross section of a 4D universe containing our “stack” of cubes. We can re-position our universe on the stack (as you could re-position a 2D plane making a cross section with a cube to obtain a non-square cross section). At each new position, we might see a cube. If we went too far in the new dimension, we would go “off the edge” and see nothing. If we rotated our universe (as you might tilt the paper to get a rectangle or triangle), you would see something else.

One more. You learn to draw prisms by drawing a 2D shape and projecting lines off its vertices to create a 3D shape. If you draw a square and project lines from each corner, you can draw a cube. At the end of each line is a vertex of a second square, located in a different 2D plane of the 3D space. Although you’re drawing on a two dimensional surface, you imagine these lines as being projected in a third dimension to create an image of a 3D shape. So take a cube, and from each vertex, project a line through a new, fourth dimension. At the end of each line is a vertex of a second cube, located in a different 3D space of the 4D “hyperspace”.

Hyperspace...but not like that

An Example

Continuing from the last analogy, suppose, rather than projecting from each point to points on a second identical cube, we project to a smaller cube. This is the four dimensional equivalent of a square frustum. I guess we could call it a cubic frustum. Now, if this cubic frustum was to pass through our universe, starting with the larger “end”, we would first see a large cube appear. The cube would gradually shrink, and eventually vanish. If, on the other hand, the frustum passed through slightly crookedly, we would see a pyramid which grew into a distorted cube-like shape which got smaller and smaller and eventually turned into a pyramid and then disappeared. This also demonstrates how thinking of the 4th dimension as time can help us visualize four dimensional shapes.

The common image of a tesseract

The Tesseract

Here is a demo. Use WASD to move, left/right to turn, and play with the number keys and “-”/“=” keys to rotate it.

A tesseract is the next progression in the “point, line, square, cube” sequence. It is a four dimensional shape obtained by projecting lines from a cube through a fourth dimension. I said that in our 3D universe we can only see 3D “cross sections” of 4D shapes, so you should be asking yourself “Why does a tesseract look like a small cube inside a big cube?”.

Here’s a quick story about how I learnt the answer to that question. The tesseract demo was initially intended to be a 4D shape exploring demo. It would display a 3D cross section of a 4D shape, and let users move around in four dimensions. Here is said demo. It has the same controls as the tesseract demo, and i/k moves in the fourth dimension. The 4D shape in the demo is a “cube based hyper-hourglass”. That is, a cube projected to an inverse cube, so the centre is a single point.

But I digress. I was working on that app, and got it to the stage it’s currently at. I’d been reading some literature on 4D shapes, and I’ve read about tesseracts in the past, and whenever you see an article about tesseracts, you see the picture of the little cube inside the big cube. Until now, I never really understood why it’s drawn like that. It’s a 4D shape, how can they even fit the whole thing in a 3D space, and why draw it like that.

But it’s actually really simple. Let’s step things down a notch. Look at a picture of a cube, drawn on paper. The cube is 3D, but the image is 2D. Also, if you look at the cube from the right angle, it looks like a small square inside a big square. The difference in the size of squares is because of perspective. Things in the distance look smaller. Now, in the case of the tesseract, perspective is working in another dimension. When you look at the small cube in the big cube, the inside cube isn’t actually smaller, it’s just further away. It’s further way in the fourth dimension.

That was the epiphany that lead to me being able to create the tesseract demo. I already had the framework in place to convert 4D shapes into 4D shapes by taking cross sections. It was a simple modification to convert 4D shapes into 3D shapes using projection.

The Demos

Hyperspace

I have already written a 3D engine in javascript. In fact I have written several, but the most advanced one is this. A 3D engine takes a description of a 3D environment and converts it into a 2D image which can be displayed on a screen. The aim of this project was to display a 4D image on a screen. It was easiest to write a program that converts a 4D environment into a 3D environment, and let my 3D engine to the rest of the work. So that’s what I did. The interesting part is how I convert a 4D environment into a 3D environment. For this demo, I used cross sections. The 3D to 2D analog is using a plane to cut a 3D shape, then displaying the cross section of the plane and the shape on the screen. I take a 3D cross section of a 4D shape, and hand the result to my 3D engine.

Tesseract

The difference between this and the hyperspace demo is the way it converts a 4D shape into a 3D shape. Rather than taking cross sections, this demo uses perspective projection. I multiply every vector describing a vertex by the reciprocal of the distance from that vertex to the viewing position in the fourth dimension only. This is why things further away in that dimension appear smaller. The entire 4D shape is draw inside the 3D space in the same way as a 3D scene can be drawn on a 2D canvas.

Philosophy Time!

Back to the idea of thinking of the fourth dimension as time. The universe progresses over time, according to the laws of physics. Ultimately, this bring about changes of state over time. A tree will grow, a mountain will erode, a person will age. A model could be proposed where an object’s existence was mapped out in a fourth dimension. Each 3D cross section of the object represents the object at some point in the course of its existence. Over time, we are just progressing along the 4th axis at some rate, and so the 3D cross section we perceive of the 4D universe we occupy is constantly changing.

Vertices, Edges, Faces, ???

A point can be thought of as a vertex. A line has two end points which we could consider vertices, and the line itself could be considered an edge. A square has vertices and edges, and the square itself could be considered a face. A cube has vertices, edges and faces.

What about a tesseract? A cube has 8 vertices, and since a tesseract is obtained by connecting two cubes, it has 16 vertices. A cube has 12 edges. The tesseract has two cubes worth of edges, and an additional edge for each vertex of a single cube, which are used to attach the two cubes, so a tesseract has 32 edges. A tesseract has 2 cubes worth of faces, plus an additional face for each edge of a single cube, which connect the corresponding edges in each cube, so 24 faces.

Say I represent a tesseract by specifying its vertices, edges and faces (which is exactly what I do in my demos). If I take a 3D cross section of it, at any point apart from its ends, I will only get a 1D cross section of its faces. Each of its faces can be considered a subset of a 2D plane, which may or may not lie inside a given 3D space. The possibilities are the plane is parallel to the space, and doesn’t enter it at all, it’s parallel to the space and lies inside the space (the only case in which the entire face is visible as a 2D entity), or the plane intersects the space along a line. The third case is the one which occurs in the middle of a tesseract represented with vertices, edges and faces. The 1D cross sections of the 2D faces which connect each edge of the first cube to the corresponding edge of the second cube forms a wireframe cube, which is why in the hyperspace demo, before any rotations, the you see a wireframe cube.

But what if I want to see a cube with faces at any point along the teserract. If a tesseract was to pass through our universe, surely we would see a solid cube, and not just a wireframe. But how would we make it so that a cross section of a tesseract has actual faces? In addition to the vertices, edges and faces, we need something else in our tesseract representation. Let’s introduce the “hyper-face”. This is actually just a 3D solid. In this case it is a cube. 6 cubes to be precise. For each face of the first cube making up the tesseract, there is a hyper-face (cube) with that face as one of its faces. Its face on the opposite end is the corresponding face of the other cube making up the tesseract. Its other faces are those connecting the corresponding edges of each cube which surround its two existing faces. The cross sections of these hyper-faces are 2D faces, which lie in the right places such that the 3D cross section of a tesseract defined with hyper-faces is a cube with 2D faces.

Agda Programming Environment

Edit:

A few things to note:

my agda-helper program is obsolete, given that “agda -I” works (though it might say that it’s not supported). :typeOf is equivalent to the :t in agda-helper.
the problems getting emacs’s agda-mode to work properly was due to a version mis-match between the Haskell Agda library and the Agda I installed with apt-get.
to fix everything, build Agda from source or find an up to date binary rather than using apt-get

Original Post

Agda is a dependently typed, functional programming language. It bears some syntactic resemblance to Haskell, though I am told that once I actually start using it, I’ll find this resemblance to only be skin deep. This post is not about the Agda programming language, but about the development environment one uses when programming in Agda. That is, it’s about the tools used to write, compile, check and test Agda code. More specifically, it’s about my experience setting up these tools on my computer. I’m posting this because the experience was somewhat tedious, and so knowledge of it may help others make their experiences less tedious. It also resulted in the development of a simple tool that might make some people’s lives easier when testing Agda programs, but more on that later.

Installing Agda

For the record, my box runs 64 bit Ubuntu 12.04. The first thing I did was:

% apt-get install agda

So far so good. Next, I installed the Agda libraries for Haskell. “Agda libraries for haskell?” you ask. Operations on Agda programs are done through a Haskell interface. For people new to Haskell, it has its own package manager called “cabal”.

% cabal install agda agda-executables

And cross your fingers. Cabal has always been very touch and go in my experience, but after re-installing ghc (the Haskell compiler) and deleting my “~/.cabal” directory several times the, gods of Haskell decided to grant me with a working install of the Agda-2.3.0.1 library (the version compatible with ghc7.4.1 which I happen to run). I know there was probably a more elegant way to install that, but meh - it worked. For people treating this as instructions for setting up Agda, I recommend you at least try to install the Haskell library without doing what I did.

Hello, World?

Not exactly. Once Agda was installed, I had a nice little “agda” binary that was, among other things, an Agda compiler. I wrote a little Agda program to make sure everything worked. It wasn’t the traditional first-program-in-language, that just prints out “Hello, World!”, because in Agda, like in most functional languages, the emphasis is on defining equations rather than doing things, and printing out a string is doing something so is actually non-trivial. Instead, I defined a simple type:

-- file: hello.agdamodule hello wheredata Blah : Set where    a : Blah

Without getting into too much detail, this code defines a type “Blah”, and a data constructor “a” that is a member of that type. It’s worth noting that this will not compile, since it lacks a “main” function (which is what the compiled program does when it’s run). But it took me a while for me to figure out how to get that information.

I’ll highlight my thought process. First I looked at the documentation that came with Agda:

[/home/steve][steve@stevebox] $ agda --help                                                                                                                                                                                                    [19:20:24]AgdaUsage: agda [OPTIONS...] FILE  -V      --version                                   show version number  -?      --help                                      show this help  -I      --interactive                               start in interactive mode  -c      --compile                                   compile program using the MAlonzo backend (experimental)          --epic                                      compile program using the Epic backend          --js                                        compile program using the JS backend          --compile-dir=DIR                           directory for compiler output (default: the project root)          --ghc-flag=GHC-FLAG                         give the flag GHC-FLAG to GHC when compiling using MAlonzo          --epic-flag=EPIC-FLAG                       give the flag EPIC-FLAG to Epic when compiling using Epic          --test                                      run internal test suite          --vim                                       generate Vim highlighting files          --html                                      generate HTML files with highlighted source code          --dependency-graph=FILE                     generate a Dot file with a module dependency graph          --html-dir=DIR                              directory in which HTML files are placed (default: html)          --css=URL                                   the CSS file used by the HTML files (can be relative)          --ignore-interfaces                         ignore interface files (re-type check everything)  -i DIR  --include-path=DIR                          look for imports in DIR          --no-forcing                                disable the forcing optimisation          --safe                                      disable postulates, unsafe OPTION pragmas and primTrustMe          --show-implicit                             show implicit arguments when printing  -v N    --verbose=N                                 set verbosity level to N          --allow-unsolved-metas                      allow unsolved meta variables (only needed in batch mode)          --no-positivity-check                       do not warn about not strictly positive data types          --no-termination-check                      do not warn about possibly nonterminating code          --termination-depth=N                       allow termination checker to count decrease/increase upto N (default N=1)          --no-coverage-check                         do not warn about possibly incomplete pattern matches          --type-in-type                              ignore universe levels (this makes Agda inconsistent)          --sized-types                               use sized types (inconsistent with coinduction)          --injective-type-constructors               enable injective type constructors (makes Agda anti-classical and possibly inconsistent)          --guardedness-preserving-type-constructors  treat type constructors as inductive constructors when checking productivity          --no-universe-polymorphism                  disable universe polymorphism          --universe-polymorphism                     enable universe polymorphism (default)          --no-irrelevant-projections                 disable projection of irrelevant record fields          --experimental-irrelevance                  enable potentially unsound irrelevance features (irrelevant levels, irrelevant data matching)          --without-K                                 disable the K rule (maybe)Plugins:

That’s actually it. There is no man page for this program. The first bit of that that looked relevant is the “-c” option which compiles an Agda program. This was before I realized the necessity for a main function. So I ran:

$ agda -c hello.agda

The resulting error message explains that I need a main function. Good we’re getting somewhere.

Then I tried importing a standard library function into my agda program. I added this line to the start of the module:

open import Data.Char

and compiled:

[/home/steve/src/agda][steve@stevebox] $ agda -c test.agda                                                                                                                        [19:39:35]Checking test (/home/steve/src/agda/test.agda)./home/steve/src/agda/test.agda:3,1-22Failed to find source of module Data.Char in any of the followinglocations:  /home/steve/src/agda/Data/Char.agda  /home/steve/src/agda/Data/Char.lagdawhen scope checking the declaration  open import Data.Char

Pretty easy to figure out what’s going on here. The compiler doesn’t know where to look for the standard library. I had a wild stab in the dark:

[/home/steve/src/agda][steve@stevebox] $ ls /usr/share/agda-stdlib                                                                                                                [19:41:35]Algebra/   Foreign/    IO/        Algebra.agda      Coinduction.agdai  Induction.agda   IO.agdai     Record.agda      Reflection.agdai  Universe.agdaCategory/  Function/   Level/     Algebra.agdai     Function.agda      Induction.agdai  Level.agda   Record.agdai     Size.agda         Universe.agdaiData/      Induction/  Relation/  Coinduction.agda  Function.agdai     IO.agda          Level.agdai  Reflection.agda  Size.agdai

(I tab-completed from “ag”.) Yay I found it! Looking back at the usage of “agda”, I use “-i” to add import paths. So:

[/home/steve/src/agda][steve@stevebox] $ agda -c -i /usr/share/agda-stdlib test.agda                                                                                              [19:41:40]The name of the top level module does not match the file name. Themodule test should be defined in one of the following files:  /usr/share/agda-stdlib/test.agda  /usr/share/agda-stdlib/test.lagda

Huh? It’s looking for my source file “test.agda” in the import path. This story will be resolved later, but first, a tangent:

Enter Emacs

It was around this time that someone on the IRC channel on which I was asking for help asked if I was using emacs (I wasn’t) and I asked if I should be (and the reply was “yes”).

Emacs is a very extensible text editor with many powerful tools to help programmers. Some extensions exist for various languages that add tools for testing and compiling code from in the editor. One such extension exists for Agda, and it was installed when I apt-got Agda (but all the documentation I read said I should have gotten a “agda-mode” binary with Agda (I didn’t) and when I run that it will install agda-mode in emacs (the name of the agda extension)).

At this point I’ll make readers aware that I use VIM as my editor of choice. I have nothing against emacs, I just happened to be around lots of VIM users when I outgrew gedit (another editor).

So I quickly learnt the basics of emacs. This was a bit frustrating because I’ve spent the better part of the last three years building up muscle memory for VIM shortcuts (dvorak VIM shortcuts mind you), and having to learn an entire new set of commands with lots of Control and Meta key presses was…unpleasant. Nevertheless, emacs’s agda-mode gave me a nice drop-down menu of things to do with Agda code (including compile, evaluate an expression and find the type of an expression).

So I tried compiling my code with the standard library import. I got an error message:

Problem encountered. The *ghci* buffer can perhaps explain why.

(ghci is the name of an interactive Haskell interpretter.)

Nice. This was before I realized that Agda operations are performed through a Haskell interface, and before I learnt all about buffers in emacs, so I was pretty confused by this. I found out that buffers are effectively containers for text. When you open a file in emacs, its contents is copied to a buffer. When you save a file, the buffer is written to the file on disk. Buffers can also exist for non-files, and the ghci buffer is one such example. Since Haskell is used to compile Agda, a ghci session is run which can be accessed in a ghci buffer. It allows you to interact with the ghci shell just as you normally would in a terminal. After the failed compile attempt, I had a look at the buffer:

HCi, version 7.4.1: http://www.haskell.org/ghc/  :? for helpLoading package ghc-prim ... linking ... done.Loading package integer-gmp ... linking ... done.Loading package base ... linking ... done.Prelude> :set -package Agda-2.3.0cannot satisfy -package Agda-2.3.0    (use -v for more information)Prelude> :mod + Agda.Interaction.GhciToplocation info>:    Could not find module `Agda.Interaction.GhciTop'    It is a member of the hidden package `Agda-2.3.0.1'.Prelude> ioTCM "/home/steve/src/agda/test.agda" Nothing ( cmd_write_highlighting_info "/home/steve/src/agda/test.agda" "/tmp/agda2-mode8592JyE" )<interactive>:4:1: Not in scope: `ioTCM'<interactive>:4:50: Not in scope: `cmd_write_highlighting_info'Prelude> ioTCM "/home/steve/src/agda/test.agda" (Just "/tmp/agda2-mode8592jGR") ( cmd_compile MAlonzo "/home/steve/src/agda/test.agda" [".", "/usr/share/agda-stdlib"] )<interactive>:5:1: Not in scope: `ioTCM'<interactive>:5:74: Not in scope: `cmd_compile'<interactive>:5:86: Not in scope: data constructor `MAlonzo'

Alright. So something is going horribly wrong. This first thing that fails (well, the first thing it’s trying to do) is setting the package to “Agda-2.3.0” (the Agda library package for Haskell I installed at the start). But hang on. The version of that package was 2.3.0.1, not 2.3.0. At some point, wires are getting crossed about which version of Agda to load, and the wrong one is being attempted and failing because it doesn’t exist. Ok, so this is an interactive shell, so I can fix this. At the ghci prompt, I ran:

Prelude> :set -package Agda-2.3.0.1package flags have changed, resetting and loading new packages......

Lookin’ good! Let’s try compiling. The ghci buffer says:

Prelude> ioTCM "/home/steve/src/agda/test.agda" (Just "/tmp/agda2-mode8592Xvp") ( cmd_compile MAlonzo "/home/steve/src/agda/test.agda" [".", "/usr/share/agda-stdlib"] )<interactive>:5:1: Not in scope: `ioTCM'<interactive>:5:74: Not in scope: `cmd_compile'<interactive>:5:86: Not in scope: data constructor `MAlonzo'

Aww man! Ok so ioTCM is a function for doing Agda magic, but it’s not currently in scope. Loading the package made ghci aware of the existence of some Haskell modules, and ioTCM resides within one of them. In ghci, there is a “:browse” command that lists the functions in a module, and this coupled with some nice tab-completion inside ghci means I can navigate around the modules inside the Agda package. But this doesn’t really help me find the function quickly.

Here’s my solution. Forget about Haskell for a second. Even if these Haskell modules are compiled code, in order for them to be of any use, people have to be able to call functions from inside them using a function call in source code. For this to work, the name of each function probably exists in plain text somewhere in the module file which contians it. And how do we find a string inside a file inside a directory structure? GREP!

[/home/steve][steve@stevebox] $ grep -rn 'ioTCM' ~/.cabal/lib                                                                                                            [20:29:16]Binary file /home/steve/.cabal/lib/Agda-2.3.0.1/ghc-7.4.1/libHSAgda-2.3.0.1.a matchesBinary file /home/steve/.cabal/lib/Agda-2.3.0.1/ghc-7.4.1/Agda/Interaction/GhciTop.hi matchesBinary file /home/steve/.cabal/lib/Agda-2.3.0.1/ghc-7.4.1/HSAgda-2.3.0.1.o matches

Once again, possibly not the most elegant way to solve the problem but it works and it works fast. This reveals that the module is “Agda.Interaction.GhciTop”. So let’s import it!

Prelude> import Agda.Interaction.GhciTopPrelude Agda.Interaction.GhciTop>

And compile again. The long pause means that it works. It will have compilation errors, but at least the compiler is being invoked. Success.

Now let’s do something useful

…where useful is context sensitive and this is the context of learning Agda. When I was learning Haskell, the most valuable tool by far was ghci (yes, the prompt that I just fought to get my Agda to compile). It lets you enter Haskell expressions and it will print out what the expression returns. You can define functions, and then call them. It lets you load a Haskell source file and use functions and types defined in that file in the interactive shell. Here’s an example:

Prelude> concat ["hello", "world"]"helloworld"Prelude> 1 + 23Prelude> let incr = (1+)Prelude> incr 56

Haskell is strongly typed, and as a result, it’s much easier to encounter type errors than in say, perl. Sometimes, it’s useful to confirm the types of expressions with Hasell’s type checker before using them. ghci can do this:

Prelude> :t concatconcat :: [[a]] -> [a]Prelude> :t (+)(+) :: (Num a) => a -> a -> a

As more information gets encoded in the type system, this tool becomes more and more useful. In Agda, the type system is more powerful than in Haskell, and more information can be encoded in it. As such it would be very useful to have a similar tool. I asked if such a tool was available on the IRC channel and was told that in emacs, C-c C-d can be used to check the type of an expression. Similarly, C-c C-n can be used to evaluate or “normalize” an expression.

But I missed the way it worked in ghci. In emacs, I can’t see my history as I enter new expressions to infer types or normalize. Doing it all through emacs just felt too constrictive.

Remember how when I did anything Agda the Haskell function being called would be sent to the ghci buffer. When I use emacs to infer types or normalize, I can see the Haskell function call being made. I jumped into my own ghci shell in a terminal and tried to reproduce the results:

Prelude Agda.Interaction.GhciTop> ioTCM "/home/steve/src/agda/test.agda" (Just "/tmp/agda.tmp") ( cmd_load "/home/steve/src/agda/test.agda" [".", "/usr/share/agda-stdlib"] )...Prelude Agda.Interaction.GhciTop> ioTCM "/home/steve/src/agda/test.agda" Nothing ( cmd_compute_toplevel False "a" )agda2_mode_code (agda2-status-action "Checked")agda2_mode_code (agda2-info-action "*Normal Form*" "a")agda2_mode_code (agda2-goals-action '())Prelude Agda.Interaction.GhciTop> ioTCM "/home/steve/src/agda/test.agda" Nothing ( cmd_infer_toplevel Agda.Interaction.BasicOps.Normalised "a" )agda2_mode_code (agda2-status-action "Checked")agda2_mode_code (agda2-info-action "*Inferred Type*" "Blah")agda2_mode_code (agda2-goals-action '())

There is a lot of superfluous information there, but the value and type of the expression “a” is displayed. Knowing the Haskell functions that get this information, I knocked together a command line program that behaves similarly to ghci for type inferring and expression evaluation.

Get it here: agda-helper

Here’s an example of its use using the hello.agda from earlier:

$ ./agda-helper hello.agda...hello.agda:aagda2_mode_code (agda2-status-action "Checked")agda2_mode_code (agda2-info-action "*Normal Form*" "a")agda2_mode_code (agda2-goals-action '())hello.agda::t aagda2_mode_code (agda2-status-action "Checked")agda2_mode_code (agda2-info-action "*Inferred Type*" "Blah")agda2_mode_code (agda2-goals-action '())hello.agda::t 'c'agda2_mode_code (agda2-status-action "Checked")agda2_mode_code (agda2-info-action "*Inferred Type*" "Char")agda2_mode_code (agda2-goals-action '())hello.agda:'c'agda2_mode_code (agda2-status-action "Checked")agda2_mode_code (agda2-info-action "*Normal Form*" "'c'")agda2_mode_code (agda2-goals-action '())

So it still prints the same stuff as before but now there is a nice wrapper. I might make a little perl script to wrap this to scrape out the meaningful information in the future.

Compiling from a terminal

I almost forgot. I was having this problem:

[/home/steve/src/agda][steve@stevebox] $ agda -c -i /usr/share/agda-stdlib test.agda                                                                                              [19:41:40]The name of the top level module does not match the file name. Themodule test should be defined in one of the following files:  /usr/share/agda-stdlib/test.agda  /usr/share/agda-stdlib/test.lagda

While I was reading an online tutorial, I came upon the solution:

[/home/steve/src/agda][steve@stevebox] $ agda -c -i /usr/share/agda-stdlib -i . test.agda                                                                                              [19:41:40]

The “-i .” tells the compiler to also look in the current directory. Don’t ask why it’s necessary.

Exit Emacs?

Not exactly. This tool is more of a compliment to emacs’s agda-mode, which provides additional useful tools for writing Agda. If you want your VIM key bindings inside emacs, have a look at evil.

Framebuffers and Coffee

This was drawn using a framebuffer. (Image from images.wikia.com)

Two discoveries I recently made that will help with my game engine project as well as make my life easier in general, are CoffeeScript and using HTML Canvas elements as framebuffers. The former is a programming language that adds some “niceness” to javascript. The latter is a technique in computer graphics that allows the rendering of an image to what is effectively an invisible screen for some further processing (or just storage) before sending it to the actual screen.

CoffeeScript

Here is the website. If you ever intend on writing more than 10 lines of javascript you should click on that link and learn some CoffeeScript.

It’s very transparent, in that most lines of javascript can be converted 1 to 1 into the corresponding CoffeeScript, so don’t worry about sacrificing functionality. It has (in my opinion) a far cleaner syntax than javascript. Pythonesque indenting is used to indicate scope, brackets aren’t required around control flow statements or function calls, there’s no need for semicolons to terminate or delimit lines.

Besides the syntactic pleasantries, there is an intuitive way of defining classes (which I guess is still just a syntactic thing if you want to get technical). The reason I highlight this point is it encourages you to write classes the “correct” way for most use cases. The way I write classes in javascript is:

function Animal(name) {    this.name = name; // field    this.sayName = function() { // method        alert("My name is " + name);    }}var bob = new Animal("bob");bob.sayName(); // opens an alert box with "My name is bob"

That seem fairly intuitive (by javascript standards at least). I have a function that creates an object which I can access using object oriented semantics (fields and methods). The “class definition” and the constructor are kinda the same thing here.

Now here’s why it’s bad. Let’s say I go and create a large number of “Animal” objects. The code for the sayName method will be duplicated for every “Animal” instance I create. This means that there is memory wasted to store the duplicated code, and time wasted to copy the code each time I create a new “Animal”.

What I should have done is:

var Animal = (function() {  function Animal() {}  Animal.prototype.construtor = function(name) {    this.name = name;  };  Animal.prototype.sayName = function() {    return alert("My name is " + this.name);  };  return Animal;})();var bob = new Animal;bob.sayName();

This works the way one would expect an “Animal” class to work. It can be accessed in the same ways as the previously defined one, but code is not duplicated. To be honest, I cheated a bit here and wrote it in CoffeeScript, compiled it to javascript and changed the output to look like code a person would write.

Here’s the CoffeeScript that generated the above code:

class Animal  construtor: (@name) -&gt;  sayName: -&gt; alert "My name is " + @namebob = new Animalbob.sayName()

The “->” denotes a function with an optional list of arguments to the left and the definition to the right. The constructor takes a name, and does nothing with it. The “@” denotes a field, so the passed in name is stored in a field, and accessed in the “sayName” method.

Canvas as a Framebuffer

A framebuffer is an array of pixels that you can’t necessarily see. It can be thought of as a regular screen (or canvas element), and all the same operations can be performed.

The reason this made me so happy when I found out about it is that I want to do something like this in my game engine. If the player is obscured from view, the thing obscuring them can be made transparent in a localised area. I got this idea from the original Fallout:

A screenshot from Fallout demonstrating cutaway walls (Image from steamaddicts.com)

Framebuffers make it really easy to do this. You just have one framebuffer on which you draw everything behind the player (and the player as well if you want). On a second framebuffer, draw everything in front of the player. Then you figure out the 2d screen coordinates where the player will be drawn. Then you create a transparent circle with that point as its centre on the second framebuffer (by reducing the alpha value of pixels). Then you just copy the framebuffers to the visible canvas, with the first one being drawn before the second one.

Another benefit of doing this is it can be used to optimize vector graphics. Most of the apps I wrote in the past use vector graphics in some way. It’s easy to just tell the canvas to draw a line between two points or fill in some polygon. Before the image can be drawn though, it must be rasterized (turned into an array of pixels). When vector graphics are used for animations, each frame must be rasterized before it’s drawn, which comes at a performance cost. This is justified usually, because the content is being dynamically generated, but for this game engine, if (say) a wall was stored using vector graphics, it’s going to look pretty much the same all the time. Rather than rasterizing the wall at each frame, I can rasterize it once when the program starts, and store the result on a framebuffer. Then, whenever I need to render the wall, I can just look at the framebuffer and copy pixels to the visible screen.

To test out this technique, and also to get the feel for accessing canvas elements from CoffeeScript, I made the mudkip demo at the top of the page. The ImageLoader helper class is for dynamically loading image files. The FrameBuffer helper class just wraps invisible canvas elements and takes care of giving it a unique name and actually creating the element at runtime. I’ve dumped all the code into the box below (it’s actually divided into files). Note that this uses jquery (which is all the dollar signs).

# effectively the main function$ -&gt;  ImageLoader.loadAsync [    "http://images.wikia.com/animalcrossing/images/a/ae/Mudkip.png"  ], (images) -&gt;    # images is an array of Image objects passed to this callback           # connect to the main canvas    canvas = document.getElementById 'screen'    ctx = canvas.getContext '2d'    # create a framebuffer    fb = new FrameBuffer        # draw a box    fb.ctx.fillStyle="rgb(120, 170, 200)"    fb.ctx.fillRect 2, 2, 212, 274    fb.ctx.fill    fb.ctx.lineWidth = 4    fb.ctx.strokeStyle="rgb(50, 80, 200)"    fb.ctx.strokeRect 2, 2, 212, 274    ctx.stroke       # draw some images on the framebuffer    fb.ctx.drawImage images[0], 2, 2, 210, 272        # copy the framebuffer onto the main canvas    ctx.drawImage fb.canvas, 0, 0class ImageLoader  # srcArray: an array of strings representing the image sources  # callback: a function to call on the array of Image objects once  # they are loaded  @loadAsync: (srcArray, callback) -&gt;        # create a blank image for each source    images = srcArray.map -&gt; new Image        # maintain a count of how many images have loaded    numLoaded = 0        # load the images    zipWith images, srcArray, (img, src) -&gt;            # when the image loads, increment the counter      img.onload = -&gt; numLoaded++      # this actually loads the image      img.src = src        # this function sets itself to be called periodically to check    # if all the images are loaded, and makes the callback with    # the loaded images as its argument once they are loaded    wait = (period, retries) -&gt;      if numLoaded == images.length        return callback(images)      if retries == 0        return console.log "timeout loading images"      console.log "."      setTimeout wait, period, retries-1    console.log "Loading images"    wait 100, 20class FrameBuffer  constructor: -&gt;        # create the unique html id    id = "framebuffer" + FrameBuffer.globalCount    # create a new canvas element    @canvasArr = ($ "", {id: id})    @canvas = @canvasArr[0]    @canvas.width = 300    @canvas.height = 300    # get the 2d context for drawing    @ctx = @canvas.getContext '2d'    # increment the count    FrameBuffer.globalCount++  # used to create unique names for frame buffers  @globalCount: 0zip = (xs, ys) -&gt;  return [] if not (xs.length and ys.length)  rest = zip xs[1..], ys[1..]  rest.unshift {_1: xs[0], _2: ys[0]}  return restzipWith = (xs, ys, fn) -&gt;  (zip xs, ys).map (x) -&gt;    fn x._1, x._2

All Grown Up

This post is to introduce a new content management system and page template for stevebob.net. The frontend and backend have been completely re-written to be consistent with modern websites. There’s also an RSS feed, and rounded corners!

The style of the old site was designed to look 8-bit, using bright colours, square corners and the pixelated fractal tree (which lives on here). The new style is designed to look more modern. The colour scheme is basically the same, but the colours are washed out. Also there is a new tree. Click on it to redraw.

Now, some history. When I started this site, it was just a static page displaying this with a header bar containing links to the several other apps I had. Once I had some more apps made, the user interface became too cluttered, and so I made a home page that had a list of apps and a description.

The problem with this was that every time I released a new app, I would have to go in and manually update the html file to contain a link to it. To amend this, I created the site that was in use up until now.

Out with the old…

I wrote a php site that looks in the directory that contains all the apps, and makes a list of apps that it displays on the home page. Each app had a “.app” file that contained a description of the app, along with what to call it, using a markup language that I defined myself in php.

Then I decided I wanted a blog. The technical details of that can be found here, but the tl;dr of it is that I ended up writing what was effectively a separate website, entirely in perl, to serve as my blog. And it worked. I had a web interface for writing content and uploading photos, that was password protected, and even had a concept of user accounts. Everything was stored in a mysql database. The exact text I entered into the interface for writing content, was stored in the database. This was so I could go back and edit the post again later and have it look the same. When the post needed to be rendered, it was processed by some perl I wrote that introduced html tags where I thought were appropriate. So I effectively wrote another markup for blog posts. This worked fine until I recently needed to embed code in a post. I used a simple approach with pre tags, but it rendered incorrectly. It turned out that every double newline was being replaced with a new paragraph and the pre tag was being ignored.

I wanted the most recent post of the blog to appear on the homepage in the content area. This is where it all really started to fall apart. Remember, the website was written in php and the blog in perl. My quick fix was to run the perl script that printed out the content of the most recent post from inside the index.php of the home page, and the output was printed in the content section.

Pretty bad huh?

Since my current projects are much bigger than the apps I built in the past, I want to make more frequent blog posts to show progress, so I needed a nicer way of creating contentt.

…in with the new

You will be glad to know that the entire content management system for both apps and blog posts has been completely re-written.

Each blog post has its own directory, containing an xml file with all the text content of the post, along with elements for describing images. There is also a html element, the content of which is literally inserted, which is useful for inserting canvas elements or iframes. Also in the directory is any supporting media, such as images or javascript files. The text content is written in markdown. Apps are a similar story. Each app directory contains an xml file with the name and url of the app.

A ruby script is used to generate the entire website, based on the apps and posts it finds. It creates a directory structure containing static html files which make up the site. Templating is done using erubis. XML parsing is done using nokogiri. Markdown rendering is done using maruku.

Also, less.js is used instead of CSS. I always complain that CSS sucks and is a pain to write. Less lessens the pain somewhat by allowing variables and mathematical expressions to go inside the stylesheet. Rather than putting div elements everywhere to have several elements with the same left indentation (say), or having to duplicate some value all through the css file, a variable can be used.

The new system is faster for page loads since the code that generates the page isn’t run every time the page is loaded. It’s safer since a bug in the website can be caught at “compile time”. Also, there is a security benefit since code injections attacks can’t be used against a static site.

RPG Engine: BSP Tree Insertion

Here’s a quick post about how visible surface detection works, in particular when there are things that move around. Firstly, say you have a bunch of polygons you want to draw, and some obscure the view of others. It’s easy to demonstrate that you can’t simply sort all the polygons into some magical order, then draw them, one after the other, in that order, drawing over anything on the canvas that was previously there. Think about the bottom of a cardboard box. There are 4 rectangles, each partially on top of the other (this drove me crazy when I was a kid). Since the last polygon you draw must be completely visible since there is nothing to obscure it, there can be no last polygon drawn, so simply sorting all the polygons will not suffice.

Enter Binary Space Partition Trees. A BSP Tree is a binary tree like data structure used for visible surface detection. To generate a BSP Tree, start with an empty binary tree, and insert polygons in the following way:

insert(p, node) if the node is empty then store p in node else if p is entirely behind the node’s polygon insert(p, right subtree of node) else if p is entirely in front of the node’s polygon insert(p, left subtree of node) else split p into pfront and pbehind such that pfront is entirely in front of the node’s polygon and pbehind is entirely behind it insert(pfront, left subtree of node) insert(pbehind, right subtree of node)

Now, for our purposes, the polygons we are going to draw all face the view point, so the view point is always in front of every polygon. Suffice it to say (and wikipedia BSP Trees if you want to know more), that in this special case, if you draw all the polygons in the right subtree by this method, then draw the root node’s polygon, then draw all the polygons in the left subtree by the same method, the correct parts of all polygons are visible.

Sounds cool - what's the catch?

The whole “splitting polygons in two” deal turns out to be a bit of a problem. Since it can occur at every stage of every polygon insertion, the total number of polygons gets really big, which slows down the drawing process, and since that is something that happens at every frame, his is bad news. The order in which polygons are inserted affects the resulting tree. In practice, to generate a “good” tree, where little splitting occurs, many trees are generated with randomized orders if polygon insertion, and the best tree is taken, stored, and just loaded when the program is run.

So far, for this program, I generated the tree on paper and hard coded it into the XML file storing the world. If the following labels are applied to walls…

…then the BSP Tree might look like…

Since BSP Trees work on the notion of some polygons being in front of other polygons, they are complicated when polygons move. This demonstration includes a movable avatar (WASD) that is included in the draw order output by the BSP Tree. It turns out there is a simple way to do this. In each frame, a copy of the BSP Tree is made, and the avatar’s polygon is inserted as per the algorithm described above. This can be done for each movable polygon.

RPG Engine: Introduction

This is the beginning of the actualization of an idea I’ve had floating around for several months now. I’ve spent the better part of the last two years making cool graphical demos and applications of questionable usefulness, and now I’m going to try to put the knowledge I’ve gained as a result to some practical use - namely, making a game. Due to the interesting technical challenges involved, I’m going to document my progress, the problems I face and my solutions for overcoming them in this blog. I’ll also use it to demonstrate different features of the engine/game as they role out.

Background

I had many sources of inspiration for what I’m planning on basing the storyline around. Though I’m still not sure exactly how it will go, it will hopefully have something to do with space, spaceships and settling on other planets. Especially that last point - I think there are some really interesting challenges involved with colonizing new planets that would be fun to explore in the video game genre.

I struggled for a while to come up with a medium to express this story. I thought about making a source mod, but I didn’t want to mess around with 3D modelling of characters, since the stock Half-Life 2 NPCs will not suffice. I’ve been playing a lot Brood Wars lately and it occurred to me that an (pseudo) isometric RPG would not be THAT hard to implement, and since all the characters, items, environments - all the graphical elements are just in the form of 2D sprites, the artwork would not be infeasible to create.

Technical Overview

Admittedly, that description was kind of vague. The truth is I’m not EXACTLY sure how a great deal of the engine will work, but these details can be worked out on the fly. In terms of how I want it to look, I’ve been looking at the classic RPG Fallout) which also uses an isometric point of view. Something that I want to add to this engine that Fallout didn’t do is the option for a real-time combat system (Fallout’s is strictly turn-based).

Demonstration

I've been hacking on this project for a few days now, and it currently stands at 554 lines of javascript (not counting comments or whitespace). Functionality so far is:

Drawing floors of given elevation
Drawing slanted floors using two points to specify orientation
Drawing wireframe walls (they obscure one another but not the avatar
Drawing an avatar
Avatar movement, including moving between adjacent floor polygons
Avatar collision detection (though it's kinda broken around corners)

Use WASD keys to move the avatar in the demo below. The colour of the walls indicates whether the avatar is in front (red) of or behind (blue) the wall. Check out the XML that generates this world here.

In the demo, it appears that some visible surface detection is already taking place. This is a cheap trick I pulled for the purpose of this demo. In this case (and I think this is true for all cases where all the polygons are vertical), there is an ordering on the walls that if they are drawn in that order (drawing over what was previously on that position on the canvas) it looks correct. (That is, no wall ever needs to be split into two as is sometimes the case when determining draw order for regular polygons.) I've figured out this order in this case, listed the walls in that order in the XML file, and the engine draws them in the correct order.

Eventually, a Binary Space Partition Tree will be generated for the walls (in fact a great deal of such trees will be generated and the best one will be chosen) once, and a new, optimized XML file will be generated with the walls listed in the correct order. The "front" side of the walls is defined as the the side facing the view point. Since this is an isometric 3D engine, the same side of each wall constantly faces the view point. The main benefit of this is that there is a simple, constant way to traverse the BSP tree. Namely, draw the right (behind) sub-tree, draw the root polygon, then draw the left (front) sub-tree. For reasons about to be explained however, it is still in our favour to keep the tree representation around.

But what about characters?

Characters can move, and this turns out to be a problem for BSP Trees. They are great for determining draw order of a static 3D scene, but fall down when things start moving around and changing. Characters can be thought of as a single polygon, which is vertical, like a wall. The polygon constantly faces the view point (that is, they always face "down" the screen). We can simply insert each character's polygon into the BSP Tree to determine the draw order of a frame.

The blue, red colouring in the demo was used to come up with an algorithm for determining draw order given a movable avatar. The idea was to use information about which walls the avatar was "behind" to modify the draw order produced by traversing a BSP Tree. This seemed to work (in theory) for rooms that had corners with angles less than 180 degrees. Once convex corners start appearing, problems arise, which is why BSP insertion is the current plan for visible surface detection with movable characters.

Existential Crisis

While I was lying in bed at 4am, unable to sleep, I started to get philosophical, as one does while lying in bed at 4am, unable to sleep. In fact I’m currently lying in bed at 4am, unable to sleep, and the philosophy is most likely the reason. Let me try to articulate exactly what is keeping me awake.

Hopefully by now everyone has thought about what would happen if they made an exact copy of themselves. The copy has the same memories and thoughts as the original, and is indistinguishable to any third parties who know the original. If the original was to die, and be replaced by the copy without anyone knowing, as far as everyone is concerned, nothing happened. It could even be arranged such that the copy didn’t know that they were a copy. But note that this won’t allow a person to cheat death. If such a copy could be made, it could coexist with the original, and so it must have its own stream of consciousness. The original doesn’t now experience the world through the eyes of the copy as well as themselves. From the point of copying, the copy is their own person, though as far as the copy is concerned, they may as well be the original. But I digress, and while this is in itself a fascinating topic, it’s not what’s keeping me up.

Let’s say that the “Stream of Consciousness” refers to your sense of self - the sense that you have free will (regardless of whether or not you actually do). Now, when you wake up, your stream of consciousness begins, and you spend the day as yourself. Then you go to sleep, and the stream of consciousness ends. The next day, when you wake up, a stream of consciousness begins again. But this is a new stream of consciousness. The old one is gone, and has been replaced. You keep your memories and personality and the things that define you, but the “self” that represents your stream of consciousness is new.

This raises the question: What happened to your old self? Also, this blurs the distinction between “Falling asleep and waking up” and “Being replaced by an exact copy of yourself”, as in both cases a new stream of consciousness is inhabiting a body and mind identical to yours. It certainly raises some dire implications about falling asleep. If when you wake up, the only thing assuring you that you are indeed yourself is your memory, a copy could wake up tomorrow, and it would effectively be you (from its point of view). Or you could have woken up this morning as a copy of your former self, and have no idea.

Nevertheless, it is late, and I have things to do tomorrow. Goodbye.

New York, Alaska

On my todo list for this trip was going hiking somewhere far from civilization with trees and mountains that made the scenery different from what one generally sees in Australia. The problems with this plan (besides the vagueness) were getting to such a location for one, and also, since I’m travelling alone there is some danger going somewhere so isolated. Unbeknown to me, my trip to Alaska would solve all of these problems.

This is a tale of incredibly good fortune in the face of terrible planning (on my part), which I am entitling:

Everything went better than expected

stevebob.net trivia: This is the first photo of me to appear on stevebob.net.

Before arriving in Alaska, my plan was:

Arrive in Anchorage
Go north

After doing a little research, I found a city called Fairbanks, the second largest city in Alaska, which is north of Anchorage. I decided I would go there, and so upon arriving in Anchorage I started looking for ways to get there. The only options seemed to be a train which would take 12 hours to get there, and 12 hours to return. I wasn’t thinking about this when I booked my flight out of anchorage about a week prior, so it would have meant I had 12 hours - between 8pm and 8am - to spend in Fairbanks. I came to a compromise: Denali National Park, which is about half way between the two cities.

So at this point my plan was:

Go to Denali (by bus)
Check into hotel
???

I booked a return trip on a bus, and made reservations with a hotel (in a town named Healy, the existence of which I was unaware of until I arrived). I was told that the hotel is 12 miles from the park entrance (where the final stop of the bus was), but I was given the number of a shuttle service that could take me there.

The next day I took the bus to the park entrance as planned. The stop is at a service station/convenience store. While I was there I met a girl who had arrived in the same bus. She was in Denali to work as a white water raft instructor, but couldn’t contact her people to pick her up. I gave her the number of the shuttle I was planning to take. We both called the shuttle and sat down to wait.

This is the point at which things got interesting. A girl pulled up in her car and asked if we wanted a ride. We weren’t going far so we figured why not. We cancelled the shuttle and hitched a ride. As we pulled out our new friend announced that she wasn’t doing anything today and asked if either of us wanted to go hiking. The girl I met at the bus stop said she had to start work right away, but I realized that I had no plan for the day so I agreed. On the way to my hotel, we picked up another hitch hiker.

The trail we were walking went to the top of Mt Healy. The view from the ground was amazing; in every direction there were snow capped mountains. The view only got better as we climbed higher. Each time we came to an opening in the trees we would stop and admire the view. My camera has loads of photos that look pretty much the same:

10: OMG MOUNTAINS
20: TAKE PHOTO OF THE MOUNTAINS
30: GOTO 10

…and the view from the top:

There was still some frost at the top of the mountain. In fact there was some frost on the ground as well.

I’m still amazed that such a random chance encounter could lead to exactly what I had hoped to get out of my trip north. My good fortune was not quite exhausted yet however.

The next morning, after I checked out of my hotel, I needed to get back to the bus stop (some 12 miles away), so I called the shuttle again. It turned out that all their vehicles were far far away, and could not get to me that day. It was now 12pm and I had to catch my bus at 1:30pm so I had a bit of time to figure this out. There was a diner close to my hotel (which turned out exactly what one would expect to find in a small country town), so I grabbed some lunch, considering the possibility that I might run into someone headed my way with whom I could hitch a ride. I found out from the waitress that there is a bus that runs once an hour that would take me where I needed to go. The nearest stop was about a 10 minute walk away, but according to the schedule the bus would be there before then. As I left the diner, thinking that I would have to hitchhike back to the bus stop, the bus I was trying to catch pulled out of a road across from the diner. I ran towards it, waving my arms, and it stopped. I spent the trip back to the bus stop catching my breath and enjoying the feeling of relief.

I don't have any pictures of my hiking buddy, but these are her fingers.

The next tale of interest happened while I was in New York. (Pardon the lack of order of these stories. They are in order of how interesting I think they are, not the order in which they occur.)

I almost had to sleep on the street

My room number at the West Side YMCA Hotel

Because of my lax prior organisation of where to stay, I ended up having to hop between three hostels/hotels while in New York because of the days that they had no vacancy. One day, while at the first of these, I sat down and booked my stays at the other two. I was using a sheet of paper with a list of hostels and phone numbers that I got from the first hostel I arrived at. (I didn’t stay there because although I thought I had made a reservation, it turned out that I hadn’t and so I had to call the first hostel I stayed at from the lobby of this one using numbers from the sheet on the morning of the day I would be checking in. I arrived in NY that morning by bus which arrived at 7am and had had very little sleep being on a bus all night. But I digress.) So I pretty much went down the sheet until I found a place that had some vacancy and ended up making reservations at the Broadway Hostel and the YMCA’s Hotel. Everything went smoothly for the former.

After checking out of the Broadway Hostel, I made my way to the Urban Oasis hostel in the really busy area near the Empire State building and the famous part of Broadway. See my mistake? Maybe the two places appeared next to one another on my list of hostels. Maybe I had dreamt of making reservations at Urban Oasis. This hostel does not advertise much, so arriving at the address one would not assume that it was there. It turned out that it was on the 10th floor of one of the buildings and you had to buzz in to get them to unlock the front door. Of course, when I got there I was told that I had no reservation. Being New York, they were full.

I was a bit concerned at this point. I definitely had reservations somewhere. I went to a nearby Starbucks to use their internet to figure this out. My original plan was to use my credit card history to see if someone had made a transaction that looked like a hotel, but they hadn’t billed me (some hotels bill for the first night when you make the reservation). Luckily, there is an online service that does reverse phone number lookups. I remembered calling them once to find out that they had a vacancy but were a little pricey, then a second time to make my reservation when there were no vacancies anywhere else. I found the number that I had recently called twice and googled it, and it lead me to the YMCA hotel. I called them to confirm that I actually had a reservation, and breathed a sigh of relief when it turned out I did. Problem solved.

View from the Empire State Building looking west at sunset

Or so I thought. I arrived to check into the YMCA hotel. I went to pay and my card was declined. Embarrassed, I checked my balance on my phone and I had enough money on my card. I told them I would go and get cash from an ATM and come back. No ATMs would accept my card either. I couldn’t even use them to check my balance. This was a new development, as I had been successfully using ATMs in New York for several days. When I got back to the hotel, I had an email from my bank with the subject: Card Fraud. Well that explained a lot. It turned out that someone had made an unauthorized deduction of $1.22 and caused the bank to block my card. I called the bank and started resolving it but then my phone ran out of credit. I couldn’t top up the credit, as it required me to use my card (see Catch 22). I got some quarters from the guy at the desk, and used the payphone in the lobby to call the bank again. International calls are expensive and of poor sound quality, but I was able to get the international reversed charges number for the bank, so I called that. This put me through to a Telstra operator, to whom I told the number of the bank and she connected me, asking them first to accept the reversed charges. Then I was able to sort the mess out (where by “sort out” I mean unblock my card and hope for the best (see Ostrich Algorithm)). So everything is working again, but one of the first things I do when I get back to Australia will be getting a new card.

Anchorage

The first morning in Anchorage I rented a bike and rode around the city. Anchorage is great. There is always a view of mountains and pine forests, and the people respond when you say “Good Morning”. I spent the entire day riding along the plentiful bike trails.

Downtown Anchorage from a coastal bike trail

A river I crossed as I rode along the coastal trail

Goose Lake

As I rode through the Alaska Botanical Gardens I was constantly reminded that I was in bear country. My favourite thing was a message at the end a Bear Information sign that was to the tune of: If you are attacked by a bear, play dead so it sees that you aren’t a threat. If it doesn’t stop biting you after a few seconds it is a predatory attack. Fight back vigorously!

But I didn’t see a single bear…

Are you bear aware?

A neat novelty of being so far north is incredibly long days. This photo is of a phenomena called Alpenglow which occurs at sunset. This photo was taken 10:45pm. (See the orange strip along the bottom of the mountains.)

New York City

I traveled from Montreal to New York via Toronto by bus. In New York, everybody is in a hurry. I forgot to get photos of Central Park, but it is huge and has a really impressive forest. I also saw Wicked on Broadway.

Empire State Building

South

East

North

WTC Memorial

Toronto, Montreal, Ottawa

I finished exams almost two weeks ago and have been spending the time as a tourist in Toronto and Montreal. Here are some photos of what I’ve been up to in the past two weeks in reverse chronological order:

Ottawa

Ottawa seems like the Canberra of Canada - they couldn’t decide which city to make the capital so they built another one just for that purpose. The main attractions are museums, but rather than spending my day in the city, I wound up here:

A flooded quarry about 30 minutes drive from Ottawa. The crane is about 50m above the water and is the highest bunjee jump in North America.

A fellow Australian whom I met in Montreal had a plan to go bunjee jumping on Friday, and having no plans of my own, I decided to join him. Let me try to put into words exactly what my experience was.

For the three days from deciding to go bunjee jumping and actually doing the jump, it didn’t fully sink in that I would be throwing myself off a crane. Not until I was standing with my toes over the edge of the platform looking at the water 50 metres below did I fully appreciate just what I was about to do. But there was no time to stand there thinking about it, because the instructor had started his countdown. “5, 4, 3, 2, 1.”

I’ve been on theme park rides that involve being dropped or otherwise rapidly accelerating, and although the acceleration of these rides may be greater than that of gravity, they don’t really compare. Since it was up to me to jump off the platform, some effort was needed to overcome the voice in my head telling me not to jump off high things. This completely rational fear of falling from a great height has evolved in humans as a survival mechanism [0] and so is more deep-set than most fears. For example, if someone had a fear of big hairy spiders that are ultimately harmless to humans, and they spent enough time being gradually introduced to just how harmless they are, it’s likely that they will eventually be rid of this fear. The fear of falling is much harder to overcome. Looking back, I feel like I was mentally disconnecting the action of jumping off the ledge with the idea of falling into the pit below, just for an instant.

But an instant is all it takes. After jumping, according to the instructor, the freefall lasts three seconds. One looses track of time while freefalling, so I’m taking his word for it. For those three seconds, the part of my brain that was saying “Don’t jump.” was now saying “I told you not to jump…stupid.” Meanwhile I experienced terror like nothing I’ve never felt before. There is something soothing about screaming as loudly as possible (which is ironic because my throat was sore for the rest of the day). Maybe it took my mind off the fact that I was falling. Once the cord started to slow my descent, I had time to collect myself before being dunked head first into the water. The cord is very elastic, and so I was then yanked back up to a height that I’m told matches the height of the second highest bunjee jump in Canada. The trip up was much easier than the trip down. I had a chance to catch my breath before the second freefall. I then spent some time bouncing upside down and swaying around before the boat picked me up and took me to the shore.

Now there is an option for a second jump at half the price of the first, so that is what I did. My experience of this jump matched almost exactly that of the first (though I remembered to yell out something stupid (“swandive”)) before jumping.

Montreal

Biosphere

This is a nature museum in Montreal using the dome designed by inventor and mathematician Buckminster (Bucky) Fuller.

Old Montreal

Toronto

CN Tower

At 533m, CN Tower was the tallest free standing structure in the world at the time of its construction in 1976.

I saw this as I walked to CN Tower and thought it looked cool.

Downtown Toronto

U of T

Centre Island

Centre Island

CN Tower from the far side of Centre Island

I rented a bike and rode around the island.

People live on Algonquin Island (a small island connected by bridge to Centre Island). The small neighbourhood is incredibly overgrown, and just beautiful.

A road on Algonquin Island

The same road a bit further down

Toronto as seen from Centre Island

Exchange: Photos 1

So I know that this is well overdue, but here are some photos I have taken around Toronto. The camera I’m using is the one on my phone, so the quality is not great (though I was impressed with its ability to take night photos, since it’s a phone and all).

U of T

Photos from both the downtown and Scarborough campuses of U of T.

CN Tower behind some buildings at U of T's downtown campus. Also there is snow.

This was the first snow I ever saw (not counting snow seen from a plane). Also, this is the view from my window. (I live on residence which is why this is in the U of T section of photos.)

This is a scenic walkway at U of T's downtown campus. I think it's called Philosopher's Walk.

Another shot of CN Tower. This is taken from the far edge of campus.

These are some trees at the Scarborough campus of U of T. I remember thinking "hey the trees look kinda neat with snow on them..." while onlookers were all like "why is that guy taking photos of the trees?".

Royal Ontario Museam and Downtown Toronto

I decided to go to the ROM, and figured it would be a good time to do some exploring.

This is the ROM. It's a very impressive building.

...rawr

This is a corkscrew-shaped tunnel made by an ancient beaver. It's a clever way to dig; you can easily climb out, and won't fall into lava if the block below you has lava under it.

While I was in Canada, I saw a moose. But seriously, this is the only moose I have seen so far.

Queen's Park

A squirrel in Queen's Park

Lake Ontario. It was getting dark, but my phone camera has a reasonable night mode.

Downtown Toronto

A boat on Lake Ontario

Ice skating rink at Nathan Phillips Square. I ended up making a spur of the moment decision to go ice skating here.

Skiing

These photos were taken at the Dagmar Ski Resort.

You can't really tell from the photo, but that side of the hill is really steep.

...and that's all for now

This is the busy part of semester, so photos will be less frequent for the next few weeks. After that, I’ll be becoming a full blown tourist and travelling around Canada and probably the USA, so there will be lots of photos.

Exchange: Canada

I’ve been in Canada for 5 days now. I didn’t post this right away because I didn’t (and still don’t) have internet where I am staying (I’m currently tethering it from my phone). I’ve been settling in, buying various necessities and getting uni things set up (internet, email, passwords, etc).

Canada is great. The people are nice and the weather is cold (but bearable). To summarize my experience so far, here is a list of the key differences I have noticed between Australia and Canada.

The key differences I have noticed between Australia and Canada

There is a tax on all items, similar to Australia's GST. It's 13% (rather than 10%) but the main difference is the prices on items don't include the tax, so it seems like everything is really cheap when actually it is not. But even with the tax, things are a little cheaper than in Australia.
There are no bus tickets (at least in Toronto). There are passes (e.g. weekly passes) but for people who do not commute regularly there are bus tokens. These are little coins which you deposit in a box on the bus.
At supermarkets (at least at the one I frequent) they do not sell normal milk. Instead there are loads of products which contain milk, and have added vitamins. I don't think any of these products are advertised as being "milk".
The walk signal when crossing the road is white and it doesn't make any noise when it's time to cross.
Walmart!

Public Transport in Toronto

I’ve gotten past the learning curve associated with a new public transport system. Now that I’m used to it, I’m going to go ahead and say that it’s much better than the one in Sydney. Here’s how it works:

As mentioned in the key differences section, fares are represented by tokens. Each token costs $3 (or you can get 5 for $13). These tokens work for both buses and trains. A single token does not equal a single trip (where a trip is defined as the period between getting on and off a particular vehicle). A single token will get you from one station to any other (provided that the time between getting on the first vehicle and getting on the final vehicle is less than 90 minutes), which generally uses multiple vehicles. This is achieved through “transfers”. Buses stop at train stations and trains stop at bus stations. To catch a bus at a bus station you walked to, you must deposit a token. To enter a train station from the non-bus-stop entrance you must also deposit a token. No token is needed to enter a train station from a bus stop, and no token is needed to catch a bus from a train station (as you either arrived by train, or deposited a token to enter the station from the outside). When catching 2 or more consecutive buses, you can take a transfer ticket from the first bus to present to the other buses. So the whole “physical token” thing is a little old fashioned but it works and unlike in Sydney if you need to take multiple buses to get where you’re going you don’t need to pay multiple fairs.

But I haven’t gotten to the best part yet: Buses and trains always arrive, and they do so at the correct times. That is a nice change :)

This is all based on my observations and a brief explanation of transfers I got from a bus driver. Please correct me if any part of my explanation was wrong.

Phone

One of the first things I did when I arrived was get my phone set up. I went with Wind Mobile. I pay $29 each month for unlimited internet, calls and SMS inside Canada. So the first thing I’m doing when I get back to Australia is leaving Telstra. Unfortunately Wind does not work on iPhones, so I now have a Samsung Nexus S. No complaints so far (apart from those brought about by me not being used to the user interface - but these don’t really count). It cost about half as much as my iPhone 3G and is much more responsive (but this is probably because it is newer - no fair comparison can really be made between them).

Photos

Sorry for not posting any photos. As I mentioned earlier, I’m tethering my internet and it’s too painfully slow to upload any photos. Once my internet at home gets fixed I will upload photos.

This Blog is Written in Perl

Let me take a minute to explain myself. Perl was not my first choice of language when I decided to write a blog website. I initially started writing the site in Ruby using Sinatra and I really liked it. Coding websites in Ruby with Sinatra is intuitive and the code is readable.

As with most web apps written in Ruby, Sinatra apps run as their own process and include a web server (such as Thin). That is, the web app includes a script that launches the app and starts a web server. This is in contrast to CGI - the more traditional approach to dynamic web content, where an app is a script that prints the HTML to be sent to the browser and the web server is a separate process (such as Apache). The benefit of the former approach over the latter approach is performance. Each time a request is made to a CGI web app, the entire program must be loaded into memory and executed. When a web app is its own process (as is the case with Sinatra apps), it is loaded when the server is started and remains in memory as long as the server runs. This streamlines the process of servicing requests made to the app, making it a faster approach to CGI.

So with this performance benefit, why did I write this site in Perl CGI?

My web server does not allow me to run my own processes meaning I could not use it for Sinatra apps. Not to be perturbed, I looked for alternative hosting options and came across Heroku. Heroku is a platform for deploying web apps written in Ruby. Apps are deployed to Heroku using a git repository. Once configured, the latest version of the app can be deployed to Heroku using git push. Apps deployed to Heroku are compiled into optimized packages known as “slugs”, boosting their performance.

In addition to the web component, my blog needed access to a database in which to store posts, and some means of uploading and storing images so I can post photos.

Apps deployed on Heroku do not have complete access to a filesystem. Apps may observe their filesystem (using commands like ls and cat file) but may not create or modify files directly. This meant I could not simply upload photos, as this requires creating files. Users of Heroku are encouraged to use third party sites for file hosting, and there are modules available that may be used to connect to these services.

Furthermore, Heroku apps must use Postgres as a database, and the only way to freely (without paying) use a database is to be part of a shared database. I modified my blog to use Postgres (it previously used Mysql) but was unable to connect to the database. Heroku provides documentation for connecting to a shared database but it is aimed at apps built on Rails. To get a database working for my blog, I made a web wrapper for a mysql database running on my web server. SQL statements were sent to the wrapper as post parameters, the statement was executed on the server, which responded with HTML containing the result of the query which could be passed by the Ruby app on Heroku. This worked, but it was abandoned as it seemed insecure and very hacky.

At this point I was frustrated by all the restrictions being placed upon me by Heroku. I could write a blog in Ruby using Sinatra but I would have no way to deploy it. I didn’t want to find other sources and just wanted to get something online, so I opted for rewriting the blog as a CGI app. Writing CGI apps in Ruby is discouraged [0], possibly due to the language’s poor performance when compared with other scripting languages [1]. I chose Perl because I know it better than the other candidate scripting languages (such as Python or PHP).

This was either the biggest or second biggest Perl project I have undertaken, and it was my first time writing object oriented Perl. In Perl, most object oriented concepts (such as defining classes, creating instances of classes, extending classes, calling methods on objects) are possible, however do not feel as elegant as they do in other object oriented languages I have used (Java, Ruby, Python). Needless to say, I didn’t get into Perl for the syntax. Once you get past the syntax however, Perl’s object oriented features are quite usable. The main use of OO in this blog is elegant handling of database records (by treating them as objects with getters and setters for fields) and grouping together related functions (by making them all methods of a class). For these purposes, I found Perl’s object oriented features to be adequate.

Camelia - the spokesbug for Perl 6

The current version of Perl is Perl 5. Perl 6 is currently in development, and will be a complete re-write of Perl in which the entire Perl community is invited to take part. A glance at its Wikipedia article shows examples of static types, improved object oriented syntax and infinite lists (resembling those found in Haskell). There is currently no official release date on Perl 6, however there are several implementations of the language.

[0]http://www.dreamincode.net/forums/topic/259314-development-server-fpr-ruby/

[1]http://xodian.net/serendipity/index.php?/archives/27-Benchmark-PHP-vs.-Python-vs.-Perl-vs.-Ruby.html