Space Invaders

Iconic Space Invader

We invent games to build worlds, often founded on reality. Chess is an ancient example. The chess pieces represent real-world characters from the time of the game’s creation. The medieval world had fighting kings and queens and knights on horseback and… bishops? As a child, I found it strange that there were bishops on the chess battlefield, not realising that the church used to be a more militant power.

Bishop Invader

Games let us replay history and simulate what-ifs and what-might-have-beens.

Computer games also build worlds, in ever more immersive ways. They are simulations of their time. The seminal video game Space Invaders was released by Taito in 1978. A genre-defining shoot ‘em up, it introduced the concepts of multiple lives for the player and high scores.

Space Invaders screenshot

The creator of Space Invaders was Tomohiro Nishikado. This guy was driven: for over a year he designed and built every part of the game himself. He built his own hardware and composed the sound effects, programmed the game and designed the artwork.

Space Invaders was released around the time of Star Wars, not long after the race to the moon. People believe the game is about defending Earth from waves of invading alien creatures from another planet. It’s not. The game is about an earlier time.

Tomohiro Nishikado was born in the city of Osaka, Japan in March 1944.

Destructio ab Alto

Seventy years ago, just before midnight on 13th March 1945, forty three B-29 super-fortress bombers from the US 314th Bombardment Wing (Very Heavy) arrived in formation in the night sky over Osaka. Tomohiro was just coming up to his first birthday. The US air force had control of the skies so the bombers could fly at the low level of 2,000 metres. The planes dropped hundreds of one ton bombs onto the houses below. Napalm and incendiary cluster bombs were dropped to spread fire over a wide area.

The Osaka civil defence system was no match for the attack. Ground-based anti-aircraft guns had been set up to defend against small groups of planes, not wave after wave filling the night sky. One hundred and seven bombers from the 313th Bombardment Wing (Very Heavy) formed the next waves. One hundred and twenty four bombers from the 73rd Bombardment Wing (Very Heavy) made the final attacks. 1,732 tons of bombs were dropped in three hours, destroying around 8 square miles of the city centre.

Osaka after air raid

Thousands of Osaka residents were killed and many more were injured. Over half a million people were made homeless. Two US aircraft were lost, one on take-off.

There were more heavy raids on the city in June, again with hundreds of B-29s, and then another on 14th August, 5 days after Nagasaki. A total of 10,270 tons of bombs (92% of them fire-starting) were dropped on Osaka.

space monsters

Tomohiro’s city was destroyed from above. He named his unwinnable, city-defending simulation “Space Invaders”, in English, not Japanese. The high scores and multiple lives reflect the methods of counting the human cost of the bombings. The B-29s were the aliens who invaded his space. Americans were the Space Invaders.

Osaka Invader

How your bank is tracking your phone

I strongly suspect that my credit card company and my bank have access to my mobile phone location. Here’s why:

pa-6820617

A few weeks ago my wife was driving me to the train station. I usually drive to work but sometimes get the train. On the way we got a flat tyre and managed to limp along to the garage opposite the train station. My wife hadn’t brought her phone or purse with her so I gave her my credit card to pay for a new tyre, while I caught the train. She knew my PIN and had used it several times before to pay for meals and such without any problem, though always while I’d been there. She managed to pay for the tyre with no problems.

That morning, and for the rest of the day, I didn’t use any other payment cards to remove the chance that the credit card company and bank might spot me paying in two places at once.

Special, secret ways

The next day (so first-class post, probably posted in the morning soon after the tyre payment) I received a letter from MBNA credit cards. I knew why they’d written before I’d opened it. It said that they had issued a new card because my current card was no longer to be trusted. I don’t remember the exact phrasing, but it didn’t say what the problem was, just that they had special, secret ways to detect things and I should no longer use my card and wait for the new one. It arrived a few days later and had a whole new number which meant I had to change all my online payments.

I’d used my card over the years at some strange times of day and in a wide variety of places, but I’d never had a card cancelled before. I couldn’t fathom how they’d worked out that it wasn’t me using the card. The card companies have long been proud of their clever, secret algorithms that can spot fraudulent usage patterns, and I’d assumed they also took into account the usage location from their card machines: using a card in London at 10:00 and then again in Madrid at 11:00 would probably trigger an alert. But my card was only used once that day. I’d driven to the train station many times over the last few years, so my paying for a new tyre at that time of day wasn’t suspicious. I wondered about CCTV cameras in the garage and how male/female face detection might be happening, but ruled them out: the place wouldn’t even have cameras.

Aha

It was a few days later, watching my debit card being processed, that I realised what they must have done. They’d tracked my mobile phone and linked it to the card! My phone went with me on the train. At the time the tyre was being paid for, the phone was probably in the office with me, which meant it couldn’t be me using the card near the train station.

How?

I’ve been thinking about how they might be doing it. So, not this, but something like this

Pattern Matching

Mobile phone companies track the movement of phones. It’s fairly easy then to pick out patterns and from these give very valuable hints to the card companies. For example, they could tell if a card is ‘at home’ or ‘has gone to the usual daily place’ so the card company could be fairly sure whether the phone has been left at home or is with the owner. They could also tell how long ago the phone had passed through a given area (and at what speed: by car or foot or train etc.), perhaps indicating that the card had been recently dropped or stolen and so making any payment attempt more suspicious. I think this is what happened in my case.

Linking level

I initially thought the card-to-phone linking was being done at a level above the two companies, because I’ve never had a mobile phone contract and didn’t think the phone company new my card number, and I certainly hadn’t given the card company my mobile phone number as a contact number. Or maybe the companies were intersecting the card payment times and places with all the mobiles in those areas at those times and growing a database of cards always used in the vicinity of specific mobile phones – quickly refining it to link each card to a single phone? That’s still possible, but then I remembered I had recently topped up the phone using that credit card, so the phone company had seen my card number and had already linked it to my phone. Perhaps during the phone top-up my phone number was also passed to the card company, so both parties would know both numbers.

Possible API

There are a number of ways the card company could be asking the phone company for the tracking data. I suspect they involve the phone company charging different fees for each method, but paying those fees would be easily justified by the reduction in fraud:

  1. Before payment time. They pre-emptively and periodically ask where the card owner’s phone is and keep track of it in their systems ready for when a payment is attempted. If the attempt is not in the expected vicinity, perhaps based on the phone travelling speed, suspect it.
  2. At payment time, before approval. When a payment transaction begins, they ask where the card owner’s phone is and if it’s not in the vicinity, suspect it.
  3. After payment time. Some time after a payment has been approved, they ask where the card owner’s phone was at the time of the payment and if it wasn’t in the vicinity, suspect it.

Perhaps they use a variety of these. Maybe 2 (at payment time) for higher value transactions, and 3 (after payment time) otherwise, since it’s cheaper. Also, 3 (after payment time) is less vulnerable to any slow-down in real-time processing by any of the checking systems, but risks having to approve payments before being able to suspect them based on the phone location.

But surely this is illegal

Checking at or after payment time (2 or 3) might make the illegality of finding out a person’s location moot, since the card company would already ‘know’ the person’s location from the payment initiation. This might be a strong reason to prefer this approach.

Another way, perhaps to reduce the privacy intrusion, would be to re-phrase the question. Instead of asking where the card owner’s phone is or was, ask whether it is or was in the vicinity of the transaction. The yes/no answer perhaps wouldn’t constitute giving away the location but would still allow the location checks to be made.

To link cards to the variety of phone companies, a single point of contact would likely be needed (either per country or perhaps on a larger scale). The existing card-processing body may have taken on this role. This could then either route the card questions to the appropriate phone company or, more likely, have access to a central ‘anti-fraud’ database, collating relevant phone movement details, and answer the questions from one place.

So my guess is that the phone companies ‘share’ their phone movement data with a third-party ‘security provider’, possibly with some government protection (remember how keen they were to prop up the banks in the national interest?). The payment processor would then make a simple API call to this third-party asking for a probability that the card owner is with the card at a specified place and time. The response could be “don’t know” if the phone seems to be ‘at home’ or is switched off. Or it could be 1 if the phone is very close to the given place at that time (which could be ‘at home’ for online payments from your home IP address). Or close to zero if the phone is nowhere near the place and has recently moved: getting closer to zero as the movement pattern approaches the standard pattern for that phone. Or maybe 0.5 if the phone was last tracked some time ago but within travelling distance of the place.

A better way to avoid privacy issues with data-sharing would be for the ‘security provider’ to just forward the “if one of your mobile phones has been linked to this card, what’s the probability that the owner is in this place at this time?” question to a number of phone companies at once and return any response. I think it would be easier to justify since the card company can say that they already ‘know’ the user’s location, as does the phone company, and all they are doing is checking that they both agree, with one of them legally holding a mapping of the card to the phone number internally. This type of continuous fact-confirming could also enable the phone company to build up and maintain a mapping of cards to phones based on proximity, as mentioned earlier. They could pre-tokenise the card numbers too, so the actual checking appears to be a random-looking number, a position and a time, with a single score as a response.

So are they breaking any laws? I don’t know. Maybe they don’t apply to this location ‘meta-data’ or especially not to the inferred meta-data if they’re using the “just checking we both agree and building co-incidence mappings” approach. But I’d like them to be more open about what they’re doing and their relationships. And I’d be interested to know when and how it first began.

I don’t think the Freedom of Information Act applies to these companies. The UK Data Protection Act should make them say which details they have about a person, but I know the information they have and that isn’t the thing. It’s how they’re inferring the connections between the information that matters. This sort of tracking and data-joining is inevitable, and privacy was probably only a brief interlude in the grand history of things.

Global

I don’t think that this is just a UK thing. My bank used to ask me for the dates when I was travelling abroad so I could use my debit card without them declining it. A few years ago they stopped asking, and now the latest advice on their website is:

“Our fraud detection systems are constantly verifying transactions to ensure your security when you use your debit card abroad. This means you can have peace of mind that your card is protected from unauthorised use and you don’t need to let us know before you travel.
If we think a payment could be high risk, it may be declined so that we can get in touch to check it’s genuine.

To help us provide you with the best service, please keep your contact details (including mobile number) up to date, so we can get in touch with you if we need to.”

In other words, “we have new, special, world-spanning ways that you don’t need to know about. And, by the way, we very much think that you ought to give us the mobile number that you should have with you when you use your card.” And of course, you will be taking your mobile with you everywhere.

I realise this could all be a touch of paranoia (have you seen how they’re tracking your smartphone’s wifi signal around shops?)! I should really test my theory by buying a tyre with my phone on me, and then another day handing off my phone for someone to take to my office while I buy another tyre, but maybe it needs to be a payment above a certain amount. I think I’ll wait until I need another two tyres. Until then, I’d love to hear from anyone who can corroborate any of this.

“Welcome to InfoLink, the best way to organise your life.”

         █████ █   █ █████  ███  █     █████ █   █ █   ▄█ 
           █   ██  █ █     █▀ ▀█ █       █   ██  █ █ ▄█▀ 
           █   █▀█▄█ ████  █   █ █       █   █▀█▄█ ███  
           █   █  ██ █     █▄ ▄█ █       █   █  ██ █ ▀█▄  
         █████ █   █ █      ███  █████ █████ █   █ █   ▀█

I recently found one of my first shareware programs, InfoLink, tucked away in a BBS (bulletin board system) archive.

InfoLink was a personal information manager (PIM) – they were all the rage in the early 1990s. It took the hyperlinking ideas that were just being employed by the nascent world-wide-web and let the user build and edit interlinked pages of information. I suppose it would now be called a personal wiki. It distinguished between links for people, places, things and events and had a calendar to show the events. It could call out to external programs to view and edit files or send faxes.

I designed it in 1993 while I was travelling around Europe with work. I wrote it in 1994 using Turbo Pascal, which I’d won in an AI programming competition in a Computer Shopper show in London when I was 18. The indexing in InfoLink was my first implementation of a B-tree.

It still runs (via DOS emulators in Linux for me now), and looks pretty useful I think, even two decades on.

Here’s a download of the zipped INFOLINK software. The program executable is 70K.

Here’s a shot of the main index page:

And editing it shows the link tags:

See the breadcrumbs in the Route panel, and the lovely drop-shadow!:

I have the source code somewhere on a floppy disk, and the compiler on several 5¼” floppy disks. I think it would be a challenge to build it again.

Best Thing I Never Had

Beyoncé’s “Best Thing I Never Had” is a song about her saying good riddance to her boyfriend. It has some cleverly engineered but contradictory sounding lyrics:

  1. You turned out to be the best thing I never had
  2. I will always be the best thing you never had
  3. I will never be the best thing you never had

Sentences 1 and 2 use exactly the same phrase “the best thing never had” to mean directly opposing things for You and I. While sentences 2 and 3 seem to flatly contradict each other, but mean the same thing. How come?

Let’s analyse the sentences and see how they work. To make it simpler, I’ll refer to the singer as B (for Beyoncé) and him as C.

Starting with sentences 1 and 2, the trick lies with the word “best” and how its scope can be stretched in two ways, making the phrase ambiguous.

Sentence 2 (“I will always be the best thing you never had”) has the simpler interpretation. B (Beyoncé) states that she is the best thing. The word “best” is tightly bound to the word “thing” – it has a very restricted scope. So “best” here can mean its usual ‘highest valued’. And then the “you never had” is an extra bit of information that states that C (he) has never had B. The upshot is that B is the best thing. Here it is a bit more formally:

[B is a thing ∧
 ∀x[[x is a thing ∧ ¬x = B] → B has higher value than x] ∧
 ¬C had B
]

In sentence 1 (“You turned out to be the best thing I never had”), the not-having is being done by B. The scope of the word “best” can include the “never” and then its negation affects the interpretation of “best”. So “best” here can mean the ‘best to not have’. The best thing to not have, from her point of view, is the lowest-valued thing. So, loosely speaking, C is the worst thing. This is the key to the seemingly misworded song title.

[C is a thing ∧
 ∀x[[x is a thing ∧ ¬B had x ∧ ¬x = C] → C has lower value than x]
]

Sentence 3 (“I will never be the best thing you never had”) is interpreted in the same way as sentence 1 (with the parties exchanged) and then negated by the initial “never”, and not as simply a negation of sentence 2 despite the syntactic similarity. C is doing the not-having and from C’s point of view, the best thing to not have is the lowest-valued thing. So here “the best thing you never had” says that B is the lowest-value thing that C has never had. The “I will never be” inverts this to mean B is the highest-value thing that C has never had (and adds that this state will be maintained as long as B is around). So, again roughly speaking, B is the best thing.

¬[B is a thing ∧
  ∀x[[x is a thing ∧ ¬C had x ∧ ¬x = B] → B has lower value than x]
 ]

Of course, our dual interpretation of these ambiguous sentences is made in the context of the rest of the lyrics. In isolation, we could interpret them in other ways and have one of the following:

  • a) Beyoncé as the worst thing wallowing in self-deprecation singing about her missed opportunity to have the best thing (flip our interpretations above)
  • b) Beyoncé and him both being each other’s best thing and both missing out (only use one interpretation, based on our original view of sentence 2)
  • c) Beyoncé and him both being each other’s worst thing and both lucking out (only use one interpretation, based on our original view of sentence 1)

Using the open world assumption, these ‘best to not have things’ will never do. Neither party can know the value of all the things they’ve never had, so the superlatives are moot.

 

ThinkSQL ODBC Driver Source

The ThinkSQL ODBC driver source code is now available here.

The driver is pure Delphi but the specification is C-based and very detailed. Memory handling and complex state transitions between client and server made this a large undertaking.

The Java JDBC specification was closely based on ODBC, but garbage collection made the JDBC driver much easier to implement.

It’s worth mentioning that the Python DB API specification is tiny in comparison to the ODBC documents but led to a much simpler, smaller and more elegant driver codebase. The Python driver can do all the things that the ODBC driver can do, but with less than a tenth of the code. It is more readable and maintainable and less fragile, and easier to use. And it actually works on multiple platforms. The surprising thing was how much simpler Python made the low-level network communications (mostly thanks to the struct module), especially considering Java was originally designed for low-level systems. Compare this Python method with the Java and Delphi ones:

The Python version

def getpUCHAR_SWORD(self):
   if self.bufferPtr+_sizeof_short>self.bufferLen:
       if self.bufferPtr==self.bufferLen:
           self.read()
       else:
           return _fail

   s=self.buffer.read(_sizeof_short)
   self.bufferPtr+=_sizeof_short
   si=struct.unpack('<H', s)[0]
   self.bufferPtr+=si
   return self.buffer.read(si)

The Java version

public String getpUCHAR_SWORD() { 
  if (bufferPtr+Global.sizeof_short>bufferLen) {
    if (bufferPtr==bufferLen) {
      Read();
    }
    else { //the buffer is not quite empty
      return Global.failString; 
    }
  }
  short usi=0;
  for (int siz=Global.sizeof_short-1; siz>=0; siz--) {
    usi<<=Global.sizeof_byte;
    short b=(short)buffer[bufferPtr+siz];
    if (b<0) {b=(short)(b+256);}
    usi = (short)(usi | b); //i.e. reverse order
  } 
  bufferPtr=bufferPtr+Global.sizeof_short;
  if (bufferPtr+usi>bufferLen) {
    if (bufferPtr==bufferLen) {
      Read();
    }
    else { //the buffer is not quite empty
      return Global.failString; 
    }
  }
  bufferPtr=bufferPtr+usi;
  return new String(buffer,bufferPtr-usi,(int)usi-1);
}

The Delphi (C-like) version

function TMarshalBuffer.getpUCHAR_SWORD(
                                var puc:pUCHAR;
                                allocated:SWORD;
                                var sw:SWORD):integer;
{RETURNS: ok,
          fail = probably means data is left in buffer, 
                 but not enough 
                 else errors as from Read
 Assumes:
   puc has 'allocated' space allocated by caller,
   unless -1 => allocate here (but caller must free!)
 Note:
   returns buffer data up to the allocated length
   (including 1 character for a null terminator)
}
const routine=':getpUCHAR_SWORD';
var actual:SWORD;
begin
  if bufferPtr+sizeof(sw)>bufferLen then
  begin
    if bufferPtr=bufferLen then
    begin //the buffer is empty
      result:=Read;
      if result<>ok then exit; 
    end
    else
    begin //the buffer is not quite empty
      result:=fail; //buffer overflow
      exit;
    end;
  end;

  move(buffer[bufferPtr],sw,sizeof(sw));
  bufferPtr:=bufferPtr+sizeof(sw);

  if bufferPtr+sw>bufferLen then
  begin
    if bufferPtr=bufferLen then
    begin //the buffer is empty
      result:=Read;
      if result<>ok then exit; 
    end
    else
    begin //the buffer is not quite empty
      result:=fail; //buffer overflow
      exit;
    end;
  end;

  if allocated=DYNAMIC_ALLOCATION then
  begin
    {Now allocate the space for the buffer data}
    getMem(puc,sw+sizeof(nullterm)); 
    allocated:=sw+sizeof(nullterm);
  end;

  if (allocated-sizeof(nullterm))<sw then
    actual:=(allocated-sizeof(nullterm)) 
  else
    actual:=sw;
  move(buffer[bufferPtr],puc^,actual); 
  move(nullterm,(puc+actual)^,sizeof(nullterm));
  bufferPtr:=bufferPtr+sw;
  result:=ok;
end; {getpUCHAR_SWORD}

The best a man can get?

For some reason I don’t really understand, I shave my face almost every day. I use a manual razor, currently a Wilkinson Sword Quattro-Iced-Titanium-Strontium-something or other. I did try to stick with a perfectly acceptable Gillette three-blade razor but was forced to upgrade to a four-blader by some clever, nationally coordinated, stocking systems. Whatever’s next? I’m now trying to avoid upgrading to a manual razor with a battery in, and an extra indicator to tell me to buy more blades.

For many years I put up with buying a new pack of blades every couple of months or so, as a single blade used to last me between 1 and 3 weeks (over £9 for a pack of 4 blades, which cost under £0.05 each to manufacture). But I could never stomach the adverts. The ones where men are hugging babies and doing man things, like driving cars and playing golf all set to a crescendoing power ballad. Especially the father’s day specials. So here’s my response to the bad adverts – a top tip that Gillette don’t want you to know that will save you hundreds of pounds: When you’ve finished shaving, dry the razor head on a towel.

That’s it: after every shave, make sure there’s no water left on the metal blades by dabbing them on a towel and your blades will last many, many times longer. I’ve been doing this for a few years, and I now only use about 3 blades a year. Do it – they’ll have less money to spend on their insulting adverts. Or grow a beard.

Think SQL

Several years ago, I spent a lot of time developing a relational database management system – ThinkSQL.

In a future post I’ll write down the reasons why, and probably release the development diaries. But for now, the source code for the server is available here: https://github.com/ggaughan/ThinkSQL

As I say in the README:

The source code is linear per se, but while writing it, it was an organic thing, generating great interwoven trees in the computer memory and in my head that were many-layered and that were modified and traversed, and impacted on a dynamic multi-versioned data store alongside many other threads, causing and needing deep psychological flow. The code comments are released as-is, and are often streams-of-consciousness.

 

Run a pipe2py-generated pipe on Google App Engine

Once you’ve used pipe2py to convert your Yahoo! Pipe into a local Python module, you can then host it as your own Google App Engine application. You just need a bit of boilerplate to handle the parameters and output formatting. I’ve created a sample project containing this boilerplate here: http://github.com/ggaughan/testpipe. You’ll need to include a copy of pipe2py, so it can find the modules it needs, and replace the demo pipeline references with your own. See the README for details.

Don’t flap when reversing

Top tip: when helping someone to reverse their vehicle into a small space – don’t stand behind the car and beckon them with gusto until you think they’re about to hit something and then show them your palm like some kind of traffic police.

(Instead, how about holding your arms apart to show them the actual size of the space and reduce the gap between your hands as they near the obstacle? That way they get immediate feedback and can see ahead of time how far they have left.)