Christopher Warner

When dealing with disparate systems normally you want to have a standard/universal format that each system understands. You do this because it makes transferring and usage of the data that much easier. There are numerous ways to achieve this but everyone seems to be of the agreement that XML tends to be the format standard de facto nowadays. That said, today let’s take a real life example of some work stuff I inherited.

 

Malscript

So what’s wrong with this script? Let us first state the intended goal.  There a bunch of menus that we would like to export from one place and import into another system pretty much round the clock. When a menu is updated and subsequently placed into a directory from a remote machine it should be imported into the other system. The script is executed via cron daemon every 10 minutes. Which means that within 10 minutes if a bunch of menus has hit the directory they will be processed and imported.

Lines 10-23; Not much going on here, we check to see that a directory exists. If not then we create it and then search for anything in /home/cwarner that happens to be a text file with the requirement of it also having the date 20090122 (lil smudged out). From there we print what we have found to the screen and copy the menus to a directory for processing. At this point we remove the files from the drop-off point and begin importing the batch of menus into the other system.

Lines 25-26; Here are where things get interesting; so let us take a look into this script /usr/local/krang/addons/NYMListings/bin/nyml_import_menus to see what exactly it is doing.

 

We are finding

So, let’s explain this bit of code a bit. The menus in question are in csv format because that’s what this system likes and there is an awkward line around 204 “skipping file ($metafile) because the ID ($rest_id) is not in the mapping file”. What is a mapping file?! In this system it actually maps an ID to a SLUG. Sounds like an index to you?! Yes; sounds like an index to me too! How is that index updated? Probably by another cron job somewhere which searches all of the data again and dumps everything out or you could do it by hand if you really want to have fun (haha, why do you do that? Isn’t that prone to all sorts of problems? This, you may be asking; Why yes sir and/or madam it is). So! There is a mapping file or index that matches an id to a slug and if the ID isn’t found then it prints that message. That said, assuming everything goes well we get to line 216 where we see a find method being executed. It’s trying to find the restaurant to update via it’s “slug”. A slug; meaning a bit of text like “juniors” would be a slug for Juniors restaurant etc. Why does it need to do that?! So it can find the restaurant object it needs to update based on the slug AND LOAD THE FULL OBJECT?!. Isn’t that insane? Yes; it is. From here we proceed to do a couple of checks to make sure we are the only ones with the restaurant open at this time etc. Round line 246 we see that we start processing the CSV menu as it has been dropped off and applying it into a new object that we can later replace with the restaurant that we just found. 

 

Processing...

So while we hold that old restaurant object we want to replace with a new one we are building based on the data in the CSV (comma, separated, value) file.  We then do anything else we need to do to the object so that we can print it out in a template and voila a brand new imported restaurant menu! Then we start the process ALL. OVER. AGAIN for every single updated menu! FAIL. U> RE! Not to mention all of the random other failures and god forbid it chokes and the stuff has to be re-run and look how maintainable and readable that Perl code is!

So how is this easier in Plone? Plone has a full indexing and catalog system for its objects. So why FIND something you already know is there? Just use the index like a normal human being does when they want to find a topic? Just check the index; awesome! So how many lines of code are we talking about? 

One; “index.slugname”

This concludes Example #2.

This entry was posted on Thursday, January 29th, 2009 at 7:53 pm and is filed under Advogato, Commentary. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.