Recursing For Bike Polo

This week I got to write a recursive function for the second time in my programming career. I’m always excited when this happens, and the event is so rare that it gets a blog post every time.

If you don’t know or care what recursion is, now’s the time to stop reading. Otherwise, we’re about to get technical, so pause your music and pay attention.

Recursion is a programming concept where a piece of code calls itself. It’s kinda hard to come up with a metaphor to explain this in non-programming terms – maybe think of a 3D printer that prints copies of itself. I dunno.

Anyway, I’ve been making a scoring application for my bike polo club. We’re having a tournament in January, and we’re not satisfied with the software options that are currently available.

Normal polo clubs use Podium. To use that, we have to get all our players to sign up for the League of Bike Polo and register themselves in our tournament with their teams, before the day of. The trouble is, our club hasn’t got its shit together. Hardly any of our players even know how to operate a computer, let alone sign up for LOBP. Besides that, our tournaments are traditionally mixers. That means you don’t choose your team – you throw your name in a hat and play with whoever you draw. We like it that way, because it keeps things low-key and egalitarian. The weakest player in the tournament still has a chance of scoring a few goals, and the winners are likely to be three strangers who have never played together before. It gives the Victoria players a chance to do okay, even though we’re playing against some of the best players in the world, who come over from Seattle and Vancouver.

So we don’t like Podium that much. (Though it is an excellent piece of software and a great contribution to the polo community!)

Jawn asked me to come up with something better last year. I think he asked me the year before as well. He asked a third time this year, with 4 months of run-up time, and this year I’m actually experienced enough to pull it off.

Instead of trying to master Javascript on the spot to create a fully browser-based application (that’s what I tried and choked on last time), I used the tools I’ve used on the job for the last three years – CakePHP and MySql. I use an older version of Cake, 1.3, at work, but my boss has been making noises about upgrading to 3.0 lately, and this was a good chance to learn how to use the newer version. So CakePHP 3.0 is the framework.

Setting up the framework on a borrowed server from work took a day or two. Figuring out how to create a plugin from the command line, then controllers, models, and views, took another week or so. Once the wiring was sorted out, I started on logic. Our user, in this case one Jawn Fawn, creates teams. Each team can have three players. They’re drawn out of a hat (in real life) then entered on a form (in the app). The teams are added to a tournament, and then you get a nice table with all your teams displaying.

Click a big plus sign to start a round, and you get… what?

Well, in the first version, you got a list of teams matched up at random. Winners get two points, ties get one point each, losers get nothing. So after a round, teams are sorted by how many points they’ve got, and then the teams with more points play each other and the teams with less points play each other. This makes sense to me, it’s a fair way of determining who the best player in a tournament is.

Except, that’s not really our goal. No one who plays polo in Victoria cares who the best player is. We’re here to get rowdy and drink in public. The weather is always terrible. Sometimes there’s soup, but the soup is usually cold. Showing each other up is the last thing on our minds.

No, the goal is to make sure that every team plays every other team, and that repeats happen as seldom as possible. Last week, I demoed the app to Jawn, and to my total shock, he didn’t care about my pretty ajax interface or the nice bubble display of complete matches. The first thing he asked was, “So if Big Country and the Beavers have the same score in round three…. they play each other again?”

And I’m like, “Yeah, I guess so?”

“Well that’s no good”, said Jawn. “The whole problem is that I don’t want to have to keep track of making sure teams don’t play each other too often.”

“Ok, I’ll give it another try.”

As usual in programming, the last 10% of the problem takes 80% of the development time. That’s basically what happened here, and it took another whole week to come up with an algorithm and an implementation that worked. Here it is, with lots of comments. It appears to be working, according to my tests, but I haven’t tested much. Still have to bug Jawn to kick it until it breaks. For some reason, he doesn’t drop what he’s doing and immediately start testing when I send him a demo, I have no idea what’s up with that.

Two things frustrated me for a whole day each. One was the part where the function calls itself. I was calling the function and then returning the array of matches. Instead, I need to return the function call within a conditional statement (if there are more unmatched teams, recurse…), and only return the matches when the condition is not met.

Second, I forgot the “break;” at the end of the foreach loop. It’s important. That was the “break”-through that let me finish it today.

Finally, the reason it had to be recursive in the first place. You grab the first item on the list of teams, then iterate the rest of the list until a match is found. What if no match is found? You have to iterate again. And so, you call the function again.

The tournament is set for January 21st. If you live anywhere near Victoria, I hope you can make it to see my work in action!

What cars are hot in Victoria?

I just got back from a month abroad, and I wonder if I’ve forgotten how to code. I ask around the office, how has it been? Did anything break while I was away?

Not only did nothing break, my boss was so bored that he’s been bothering the office manager and taking extra days off. I should probably investigate and find out what his deal is, but first I have a burning question.

What cars sell quickly on the used car market in Victoria, and what do they sell for?

This info is available through UsedVictoria – kind of. They have an RSS feed. So you can enter some search terms and they’ll show you the last 25 items posted with those criteria. You can’t page through the results, so that doesn’t give me any historical data. Any info I gather will start from today. But I can scrape the feed and save it in my own database, and graph it later once I have a body of data. Ok, here we go.

What’s going on: we ask UsedVic’s server for all the car ads they have with the following criteria: private sales only, priced between $400 and $6000. This returns a few hundred results, but the RSS feed will only give us 25. Oh well. I save each item in my database – the title, description, search criteria, date it was posted, and price.

The price, mind you, is in no way accurate. When I buy a car on UsedVic, I haggle. People who buy from me usually try to haggle as well. So the actual selling price is likely a couple hundred lower than what’s listed, in most cases. However, it does give you a little information. The listed price is the starting point of negotiation, so people generally won’t go see a car unless the listed price is at least in the realm of reality and their budget. If someone has set the price of a vehicle unrealistically high, the car will sit on the market for a long time.

This happened in the case of a rather spectacular Suzuki DRZ400SM that was posted last winter. It had several fancy racing upgrades, combined with an expensive but utterly tasteless metallic pink paint job. If I could have had that bike for $2000, I would have done so in a second even it was covered in swastikas and cocks. Paint is cheap. But the guy wanted $6500 for it, and the ad stayed up for something like 3-4 months. (Ask me how I know. Yes, I was checking every week. I have a problem. Don’t worry about me, mow your own lawn.)

So how to deal with the fact that we only get 25 items returned? Easy – check back every 2 hours. Usually 2-6 items are posted per hour. So I add a check to see if each item is already in my database, and if it isn’t, add it. This code is kind of inefficient. If any of the real programming nerds get hold of it, I will be embarrassed. But it’s good enough for my purposes, for now, probably.

So I wrote a cronjob to run this every 2 hours and report back to me when it’s done. Okay, cool.

Now how do we track items that are sold? It’s tough to count something that isn’t there. Not too tough though. More code:

You can also search for an ad using the exact title, and you’ll generally get back only that one exact result. Since I have the titles saved, I can do that. In my database, I have a column labelled “sold”. When the item is entered, that column is set to false.

This script gets a list of all items in my database that have “sold” set as false, and searches for them. If an item is not found, “sold” gets set to true, and the current date is recorded as well.

Listings expire after thirty days, which should help keep this script from getting out of control as it hits the UsedVictoria servers over and over again. After a few weeks, I’ll have hundreds of unsold listings in my database, and once a day, the script will request every single one of them. I may end up blocked by UsedVic pretty soon if the numbers get too high. But no worries for now.

Finally, I want the data in a manageable, bite-sized format. You can see the results at rocketships.ca/srs/scraper.

The table lists all cars that were posted and sold within the last thirty days, the number of days each was on the market, and the price requested. It’s still not as fine grained as I would like, but I think this will give me at least a vague answer to my question – which cars are hot in Victoria?

My theory: Miatas. I await hard data.

Next step is to decide what to do with the data. I had an idea about buying cars from the mainland, where they’re a bit cheaper, and flipping them locally. I feel like I could make a small profit doing this. However, to make it worth my time, I need a profit of at least $300 – that’s what I pay myself for working on the weekends.

So let’s say I bought a 1990 Miata for $1800 on the mainland and sold it for $2300 here. Those numbers are realistic, based on my experience to date.

I have to go the mainland ($18 there, $74 back), get my pedestrian ass to the seller’s house somehow (unless they’re kind enough to meet me at the ferry), check the car to make sure it isn’t shit (Honestly, what do I know? Not much, man. I’m a writer, not a mechanic.), and then the really fun part – figuring out insurance.

If I transfer my own car insurance to the new car, I will have to pay GST on the car. 12% of 1800 is $216. Big chunk of my profit gone right there. I can maybe get a temp permit, which is only $30 or so, but then I really have to get home smartly on the next boat, I don’t get to joyride the car while I’m waiting for a seller, and I can’t allow buyers to take a test drive. The insurance costs some money, so does registration, so does the plate if I haven’t organized that properly, and you have to pay a fee if you insure for less than a full year or if you want to pay monthly. Costs maybe $1200 for the full year? depending on the car and whether my points have expired yet.

There are ways around these issues, namely, lyin’ and breakin’ the law. I’ve bought and sold something like 20 vehicles over the years, and I will not claim perfect observance of the rules. But any business plan that relies on illegal behaviour is a bad idea, and out of the question according to my principles.

So out of $500 profit, I might get to keep $300, legally. But that’s not the main issue. You can only get away with buying and selling a certain number of cars per year before the government starts to get suspicious. I think it might be around 6. After that you have to get a dealer’s license, which is ex$pen$$ive. I’m not really prepared to go down that road – messing around with cars is alright for a hobby, but used car salesmen are considered the scum of the earth by most humans, and for good reasons. So I would have to find a way to respect myself for doing it, first. My friend Dylan does it by specializing in high end racecars. My friend Ben doesn’t do used cars – he’s an honest to god legit new car salesman who provides warranties. But there are others who are pretty slimy.

With that problem unsolved, I have made the data and the code publicly available. Enjoy.

Running a Query on Multiple Databases

Our company uses a CMS that we developed in-house, and there are around 60 sites running the most current version of it. That means about 60 sites with mostly identical database structures.

When I make upgrades or bug fixes to the CMS, I commit them to a core repository which is inherited by all the sites. Very efficient, write once, use over and over and over again. Then if you need custom code for a single site, you can override it locally. So that’s great for the code, but the databases, though all pretty similar, are nonetheless each their own unique entity, running on our MySql server.

When I need to change the databases, I have to change all of them individually. What a chore. Shouldn’t there be a programmatic way to do that? PHP5, Cake framework, MySql, hit me up, let me know what I don’t know. But in the meanwhile, I needed to add a single record to the Permissions table of each and every database.

The query looks like this;

Ok, cool. It checks whether the permission exists in the permissions table, gets the id if it does, and adds a record to grant that permission to the admin (#2) group.

But I got to do this 60 times in a row. No way Jose. I asked my pal Matt to help out, he being more advanced in his career than I am. Matt kindly wrote me a nice syntactically-correct statement using the sp_msforeachdb command – then I pointed out that I’m using MySql and sp_msforeachdb doesn’t exists, nor does it have an equivalent that I know of.

These problems are so basic, I’m almost ashamed to even write about them. Does everyone make such elementary mistakes? I’m positive they do, but I must be the only one who blogs about it. Anyway, Matt’s advice was to burn our 13 years of legacy code and start again on Microsoft Server. Okay pal.

Here’s what I ended up doing:

script.php:


Then I ran the contents of output.txt against my database in PHP MyAdmin, and no I couldn’t use mysql_connect or any of the other excellent PHP MySql functions that exists, because some of the databases were on my list incorrectly, didn’t have the right tables and therefore caused the script to fail gracelessly.

Is there a better way? Will you tell me?

CakePHP Concatenation

I had to concatenate two fields to search them as a single string. Another programmer created a table of phone numbers with the columns area_code (INT 3) and phone_number (INT 7) respectively. But people were searching for all ten digits as a single unit, so I needed to search both the columns as a single field.

This blog post on virtual fields and this doc about same made me think that  $virtual_fields was the answer, but it didn’t work out for me. The following solution worked well enough.

$conditions = array(
'CONCAT(NoCallList.area_code, NoCallList.phone_number) LIKE' => '%' . $this->data['NoCallList']['search_term'] . '%'
);
$this->set('items' , $this->{$this->modelClass}->find('all'
, array(
'order'=>array($this->modelClass . '.phone_number')
, 'limit'=>500
, 'conditions'=>$conditions
,
)
));

You can use recursion in real life!

They make you learn recursion during first-year intro to programming courses. Some people get fed up at that point and drop out to become car salesmen or Liberal Arts majors instead. Some people never quite get it, but bodge their way through another four years of school anyway. For those who do get it, there’s a neat “aha!” moment where the entire world suddenly makes perfect sense, and it feels like the future of your brilliant programming career, and all the wealth that goes with it, is spread out before you. “The function calls itself! Of course! That’s what they were saying all along!”

My pseudocode
My pseudocode

Then they pile another dumptruck of homework on you and you forget about it for a while. Because honestly, you don’t use recursion that often in everyday programming. It’s a neat trick, but very rarely the most efficient solution to a problem. It’s also very easy to write a recursive function that goes out of control and brings down your entire office’s network and causes your boss to come your desk and raise his eyebrows at you.

So this week when I found a legitimate excuse to use recursion at work, I was pretty excited.

The Goal

We have a mailing list about real estate. Real estate agents write down some stuff about a property listing, give the date of an open house, upload some pics, and pay 70 bucks for us to send the email to everyone who is subscribed to the list. If you want to buy real estate in Victoria, it comes in handy, I guess.

The only thing is, each email has to be approved by our secretary before it goes out. Also, since we don’t want to spam the mailing list, we have to space them out a little, so the system isn’t fully automated. Nevertheless, when a customer asks for an email to be sent out on Monday morning, they expect it to be done or get an explanation.

The secretary needs some time in the morning to approve the scheduled emails and request changes if needed, so the following rules exist for the datepicker:

  • If it’s a weekend, the date is unavailable.
  • If it’s a holiday, the date is unavailable.
  • Today is unavailable.
  • If it’s after 1pm, tomorrow is unavailable.
  • If it’s after 1pm and tomorrow is a holiday or a weekend, the first business day following the holiday or weekend is unavailable.

Stop here and code up your own solution according to the following spec.

Okay, you ready? I look forward to being made a fool of by all the people who came up with better solutions than mine. Nevertheless, here’s my answer, which took a full work day to hammer out but made me very happy.

First, I’ve got three helper functions: one returns weekend dates. One returns holiday dates. And one returns the day after a given date.

Here’s the code.

Solution

I only brought down our office network once in the writing of this class.

The function is called like this:

You pass nextAvail() the current timestamp. It checks whether that time falls on an unavailable day. If it does, the time is incremented by one day by calling $this->tomorrow($today); then nextAvail() is called again with the incremented timestamp.

And so on, until we find a day that works.

 

***Edit: Since this is somehow one of my highest ranking posts, I might as well plug my company: Radar Hill Web Design in Victoria, BC.

Fix My Bug

After wrestling with a stupid syntax error for several days, I got sick of staring at the same code for ages and started working on this.

Fix my bug!

Sometimes it’s more fun to help someone else with their code than work on your own. That’s what fixmybug is for. Put your syntax error up, and while you wait for someone to figure it out, knock one out  for someone else.

I’m amazed at how quickly and easily this thing came together. It’s pretty bare bones, but it still required a couple of years to learn the css, html, javascript, php, and mysql that went into it. 2 years ago I wrote my first Hello World in Python, and it took months of half-hearted stabbing at the command line to even get that far. Now I’ve got 98% of a diploma in Computer Systems Technology, I can build stuff like this, I think I’m ready to start working for a living. I look forward to reviewing my source code a year from now and being ashamed of it. But right now, I’m pretty proud.

Go to http://www.rocketships.ca/fixmybug and start squishing.

dasboot

 

Format a list using XML and Python

I made a nice gallery full of photographs of all the books currently on my bookshelf. In the next few months I’ll be getting rid of all of them, and it will be tough to let go of some. So this is for the memories.

By the way, I finally figured out the lightbox gallery that I was moaning about in a previous post, many months ago. I’m using it to display the pictures all nice and fancy. But there are almost a hundred of them, and that makes for some tedious html and xml coding. Fortunately, with all the education I’ve recently acquired, it was a little easier. I used a PHP script to grab the filenames of the pictures, thumbnail versions, author name and title of the books out of an xml file and spit it out in html.

The tough part was the xml file. The finished product was over a thousand lines long and I’m not typing that crap.

First, put all the thumbnails in one folder with names like booktitle-t.jpg. The large versions go in another folder with names like booktitle.jpg. Go like this…

…from the command line to get lists of all the file names.

Then use this quick python script to add the xml tags.

so that gets you this:

Awesome.  Add lines for the thumbnails, author and title as well.

Which gets you this:

Then just add the Author and Title fields by hand, unless you have a better idea. Run it like so:

I suppose there’s a proper way to write to a text file from within the script, but I never got around to looking that up.

 

 

Getting a little further with Stripe

Edit: I had some success getting it running with C#, read it about it here.

 

Ok, so now I’ve got the code for the payment form and the action for it on my server and running, and I can at least tell whether stuff is working or not (it’s not). Check it out at rocketships.ca/stripe/payment.php if you want.

A typical use case:

For this to happen I need:

A product page that contains:

  1. An item
  2. An “add to cart” button
  3. A “checkout” button

A checkout page that contains:

  1. A list of the customer’s items
  2. A “pay now” button

For these pages to work I need

  1. A database of items
  2. A method to get an item from the database
  3. A method to handle the “add to cart” button
  4. A way to list the customers items on the checkout page

And the last ingredient, of course, is someone who knows way more than I do, cause I feel super lost, and pretty sure that most people reading this from HackerNews or Proggit are just gonna laugh at me. Oh well, soldier on. You only learn the hard way.

 

Getting Started With CakePHP

Here’s a couple of things I did wrong when trying to use CakePHP, for your benefit.

Several months ago I applied at a company called Radar Hill, which uses Cake as their main framework. Naturally I set about learning how to use it as soon as possible. I didn’t do very well. I it on Rocketships, but couldn’t figure out how to make it run – there were a variety of problems that I’m not quite experienced enough to recognize.

Then I tried it locally on my MacBook. I made progress, but the database part stumped me until I found out about MAMP, which got me a little further. But it still wasn’t what you’d call a success. Finally, the job interview at Radar Hill rolled around, and I was not hired, mainly because of my inexperience with PHP. Sigh. I tried!

However, the guys who interviewed me were very encouraging, and said to keep trying and stay in touch. One, Jamie, gave me some tips on how to run it on the Mac. His instructions worked perfectly.

 

First Mistake

.htaccess files sometimes don’t copy and paste in the Mac finder, because the . at the beginning of the file name renders them invisible (except when it doesn’t).

The solution was found here:

Probably the hardest to do but uses the least amount of work and has the most potential to go wrong, that is the Terminal Method. If you open up terminal (Applications > Utilities) and type the following:

 

defaults write com.apple.finder AppleShowAllFiles TRUE
killall Finder

 

This will show you all of the hidden files and folders on your operating system. If you want to reverse the command replace TRUE with FALSE.

Then check the Cake files to make sure that all 3 .htaccess files are present – as detailed by this very helpful post on Stack Overflow. There should be one at the top level of your cake directory, one in /app, and one in /app/webroot.

Second Mistake

I was putting files in all sort of random places, trying to find a way that worked. Jamie said to unzip the Cake download into /MAMP/htdocs, replacing anything else in that directory. This is probably a dumb newb mistake, but I’m sure at least one other person has made it.

What Jamie Said

#1: Download and install MAMP:

It’s an “out-of-the-box” LAMP environment for the Mac. It’s all I ever use.

#2: Start MAMP, and then go to the homepage, which by default is:

#3: Click on the “phpMyAdmin” tab at the top. You can also get there
directly:

Create a new database and call it whatever you want – something descriptive.

#4: Download the latest version of CakePHP 2.2, currently RC1, from here:

#5: Extract the archive and put its contents into
/Applications/MAMP/htdocs (replace whatever’s there).

Open the database config file for editing, which is at (relative to htdocs):

app/Config/database.php.default

#6: In the DATABASE_CONFIG class in the file you’ll see this array,
which is the default DB config:

Change it to this:

#7: IMPORTANTLY: don’t forget to replace “your_database_name” above with
the actual name of the DB that you created.

#8: save that file as database.php, rather than database.php.default

#9: Browse to: http://localhost:8888/

If all went well you should see the ‘vanilla’ Cake installation.

– Jamie