marcua's blog

NYT_Transformer and Data.gov: Your chance for a weekend hacking project!

The awesome developers at The New York Times’ Open blog have just posted about NYT_Transformer, a tool for converting between various data formats (XML, comma-separated files) and data storage mediums (flat files, databases).

This isn’t the first time such a conversion utility has been written—Babel comes to mind. The NYT_Transformer has a few perks in its favor, however, namely that it seems to be used in heavy production at The New York Times, and that it allows you to convert between databases and flat files (nice touch!).

Written in php, the tool is geared toward web applications. An immediate thought, aside from batch jobs for various internal projects, would be to use the tool for a greater purpose: a data converter for data.gov.

While data.gov is a nice start at centralizing the directory of the U.S. government’s raw data feeds, it lacks a utility for converting between various data formats. Given that the sunlight foundation is currently running a competition to build tools on top of data.gov, this is the perfect opportunity to go meta and build a data.gov browser with automatic format conversion. That would help standardize the site a bit, and would be a nice signal for the folks at data.gov as to the file formats that people actually want for various datasets. If you’re interested in working on this project, let me know!

MIT Database Systems (6.830) TA Course Notes

In Fall 2008, I had the pleasure of TAing Database Systems with Sam Madden, Mike Stonebraker, and Evan Jones. I figured that I could take notes to help students follow the lectures while clarifying any confusing points that were raised during discussion. It would also help me avoid the embarrassment of forgetting something mentioned during a lecture and having students explain it to me during office hours:).

I decided to take notes in plain text, mostly out of laziness. This turned out to be a challenge for drawing things like query plans, but forced me to distill explanations into a conversational tone that provided an alternative to traditional diagrams.

Some students in the class told me that they benefited from and enjoyed the notes, and so I decided to open them up for reuse by the rest of the web community. The topics are as follows:

Feel free to take a read through the notes, and shoot me an email with requests, corrections, or just to let me know you read something.

For completeness, This work by Adam Marcus is licensed under a Creative Commons Attribution-Share Alike 3.0 United States License.

Enjoy!

Local Communities, Information Gathering

By way of danah boyd I found the Knight Commission on the Information Needs of Communities, which published an introductory whitepaper that asks how local communities find, digest, and react to information.

Outside of designing software, my biggest hobby is reading the news, so you would imagine that I would feel well equipped to answer how I keep up-to-date with, and respond to, local events. Reading the questions the commission is tackling gave me pause, however, since I realized that all of my sources tell me how things are over there, and very little about what’s going on around here. I keep up-to-date with the people who live next door, but only by way of the abstracted and aggregated views of the national news media.

I have tried to sign up for blogs about my area, but they are either too commercialized, or have a low signal-to-noise ratio. And even then, if Hurricane Katrina strikes the northeast, I’ll know as little today about what to do next as I assume most people did the first time disaster struck.

So the national news tells us about there. Blogs tell us about what we know we want. Twitter tells us about now. Where do we find out about down the street, at the moment, and what everyone is doing about it? It seems that we’ve built a digital infrastructure to maximize information gathering at the expense of a local infrastructure for dealing with the more basic things in life.

Making the case for Raw Data

Tim Berners-Lee’s recent TED talk on Linked Data has inspired quite a few people to ask what exactly linked data is, how it differs from data on the semantic web, and how realistic it is to assume universal and unique addressability of data items. A world with linked data would be a world with richer, more explorable data, and that notion on its own makes Tim’s talk worth viewing. The most inspiring part of his talk, in my opinion, was the one in which he got the entire crowd to loudly demand RAW DATA NOW. Given the push for more open datasets in government, and given that more websites are becoming API-providing data platforms, it is important to demand raw data where possible.

The magic behind raw data

The best thing about raw data is that almost everyone knows how it works. This means that as far as the data (re)user is concerned, the datasets are text files (or perhaps a close variant) that they can download, open in some default application, and get some immediate use out of it.

If the US Federal budget dataset is released as a comma-separated file, a middle-schooler can download the file, open it in a spreadsheet application, and sum the columns to see how much we’re spending on the Department of Education this year. A more skilled high-schooler can upload the file to Many Eyes, make a pie chart out of it, and post it to their blog. A first-year college student can write a php script to allow people to comment on various parts of that pie chart, allowing you to drill in to various slices to get a finer granularity.

With raw data, you’ve opened more people to more visualization, exploration, and discussion than was available through the original web application that acted as a firewall to your database.

Hugging the data to death

During his talk, Tim spoke about “Database Huggers,” or people who, for various reasons, hide their data away in databases. Once the data sits in a database, the publisher might provide a specific and constrained view of the data by way of a website, or they might hide it even more, simply calculating some aggregate statistic over the data and claiming, without verification, that the data has certain properties.

There are several legitimate reasons for database hugging. Some data was meant to be private—academic, medical, and financial information are all datapoints we’d prefer to keep private. We’d hope our service providers will keep it out of the hands of others. Similarly, a company might have competitive reasons for keeping information private, especially when it would be equally valuable to their competitors and not too valuable to the public—lists of customers and transaction histories come to mind. Keeping this information far from the publicly accessible web is responsible and wise.

There are other cases, however, where the data should legitimately stay open and publicly accessible. Open government initiatives will result in many datasets published by organizations that will or should exist in the public domain. Many Long Tail websites, maintained by small groups of hobbyists, probably would not mind if the datasets they generate are published in their full glory. For these types of applications, raw data is ideal.

Even in the case of datasets that should be open to the public, database huggers will sometimes disable direct access to the data, instead opting to place it in a database that sits behind an html-generating web application. Thinking that you’ve hidden your data behind HTML, thus making it safe from reuse, is an unwise assumption. In about an hour, a decent programmer can write a perl script to crawl your site and tease the data apart from the obfuscated HTML that surrounds it, reverse-engineering your database without asking for permission. In fact, there are tools that make this process easier than writing a one-off perl script. And if you think you can block the person from accessing every page on your site in a short period of time, then they will just collaborate with everyone else who wants the data, write a Greasemonkey script to collect parts of the site that they browse, and eventually collect your entire presented dataset.

Databases are not inherently evil. They provide an excellent way to store, index, and query data, but they also have a way of separating the average user from that data. Most websites, for example, do not publish a read-only username and password to their database, for fear of arbitrary queries that could easily take down their machines, or at least keep the machines busy for a long time. We should design tools to maintain the excellent services that databases have been built to provide over the last four decades, without limiting the access to the raw data when such access would be most valuable.

Are APIs the future of raw data?

There is a middle ground between the highly private datasets and the obviously open ones. Most forward-thinking organizations have realized this. They have also realized that if they have something to sell, be it in meatspace or screenspace, it’s better to release the data about their offerings to anyone that wants to use it, so that people eventually end up at their site. They do this by providing a web API to make their dataset queriable, essentially telling other software developers which questions they can answer about the dataset (query for books by author, query for restaurants by cuisine). Amazon has some APIs, as does Yelp, and you’d have to be a pretty self-loathing web 2.0 company to not provide an API over some portion of your data. So are APIs the solution? Not always.

APIs are a step in the right direction—open data is better than obfuscated data. APIs help both third-party developers and dataset publishers get more out of a dataset. They have a few drawbacks as well:

The API is an HTTP interface to your database. This means that if someone else makes a third-party application that is immensely popular, it’s your database that pays for the brunt of its popularity. You weren’t expecting a huge ramp-up in server load? Too bad.
As kind as the dataset publisher is, they can’t predict every use of the data—if they could, they already would have implemented the best use cases. If they can’t predict how the consumer/developer will use the data, they might not publish a good hook into the dataset. This would either prevent or make awkward the interaction between the third-party application and the publisher.
Building an API for a dataset makes the people who are nice enough to share their data do more work on top of designing their application. Following common REST or CRUD conventions makes this easier, but still puts the onus on the developer. As a corollary, APIs don’t change with the data. APIs are frequently revised, meaning that a change in your data requires constant upkeep of your API.

One might argue that some of the criticisms of APIs are unfair:

Saying that raw data will reduce the load on your database implies that the third party has some cache of the data, which is thus slightly out-of-date. You could imagine some sort of Comet-updated raw dataset system, but it’s unlikely for now that dataset publishers will be willing to stream live updates to third parties.
Perhaps the limited API functionality is for good reason. Amazon might never want you to be able to download their entire dataset—they don’t want to waste the bandwidth and they don’t want competitors to know exactly how many items they have on hand.
Publishing any sort of raw data will require extra work on behalf of the dataset publisher. Perhaps API-writing is the least invasive of their time?

An ideal data management tool would allow raw data publishing when possible, and make it easier to build APIs when some limited access is desirable. We should not pretend to know the point at which raw data is superior to APIs, but the point exists somewhere. It’s important to understand the benefits that raw data provides on top of web APIs, so that you can think about when it would be valuable to use.

After all this time, the answer was text files?

You’ve probably become skeptical of these suggestions. Are we really supposed to throw away decades of database research in how to properly store, index, and query reasonably sized datasets so that a middle-schooler can look at the data in a different way? Of course not. The interesting research question becomes whether we can give the user the illusion of raw data while still benefiting from database technology where possible.

That’s one research direction we’re taking within the Haystack group. With the constraint that the raw data, in human-readable text files, should always be available, we’d like to blur the boundaries between databases and data-aware webservers.

Specifically, what we plan on designing is an apache web server module that recognizes when it is serving a dataset, perhaps by taking note that it is serving a .csv, .rdf, or .json file. In such cases, the server would cook the data into a database behind the scenes. Data-aware clients (in javascript for the time being, but in the browser one day) can then query the web server about the data directly. Updates become difficult, but we can make consistency guarantees about the original raw data text files to ensure that someone can download them and see up-to-date information.

If you prefer programmatic access to the files, the module turns into a REST(, SQL, SPARQL, you favorite path language)-capable endpoint. If you prefer to get down and dirty with the data, you’ve got the text files.

We certainly don’t want to stand in the way of a world with Linked Data, so if you’d like, the tool will eventually return data with URIs. We can’t guarantee the URIs will resolve to anything useful, but that just might require a human’s touch. We’re not sure how that fits into the picture for the average data publisher, since the marginal benefit to the individual of universally addressing your own data is small, whereas the benefit to everyone else of adding another linked dataset grows with the number of datasets it is linked to.

And now, for some questions

We’re early in the development of our tools, so we’re open to your ideas and suggestions. Keeping text files up-to-date with the database that’s proxying them is nontrivial. Thinking of the ideal client/server mode of operation will also take time. We probably haven’t thought of the most important must-have feature yet, so any suggestions are welcome.

Thanks to Ted Benson, Sam Madden, and David Karger for their thoughts on this post.

(Cross-posted on the Haystack Blog)

Rethinking the Newsroom

I’ve had a lot of discussions with people about what news will look like in a digital age. Clay Shirky has just written this piece about that topic. He’s managed to say a few major things:

Big newsrooms were a means to an end. Unfortunately for them, what we actually want is information via journalism, and until recently, not everyone could afford the printing press that was a prerequisite.

“Society doesn’t need newspapers. What we need is journalism. For a century, the imperatives to strengthen journalism and to strengthen newspapers have been so tightly wound as to be indistinguishable. That’s been a fine accident to have, but when that accident stops, as it is stopping before our eyes, we’re going to need lots of other ways to strengthen journalism instead.”

It turns out that a side benefit of being a market with little competition was that advertisers could pay for your more extravagant expenses.

“For a long time, longer than anyone in the newspaper business has been alive in fact, print journalism has been intertwined with these economics. The expense of printing created an environment where Wal-Mart was willing to subsidize the Baghdad bureau. This wasn’t because of any deep link between advertising and reporting, nor was it about any real desire on the part of Wal-Mart to have their marketing budget go to international correspondents. It was just an accident. Advertisers had little choice other than to have their money used that way, since they didn’t really have any other vehicle for display ads.”

Putting information into bits on computers has made it easy to extract the bits and redisplay them without DRM or paywalls. You won’t get around that, and so the model has to change.

“And so it is today. When someone demands to know how we are going to replace newspapers, they are really demanding to be told that we are not living through a revolution. They are demanding to be told that old systems won’t break before new systems are in place. They are demanding to be told that ancient social bargains aren’t in peril, that core institutions will be spared, that new methods of spreading information will improve previous practice rather than upending it. They are demanding to be lied to.”

While well thought out, it leaves us wondering what comes next. Competition is good in that it lowers the price to entry. It’s not good in that no one entity has enough money to run the Baghdad Bureau. We’ve also heard that local newsrooms are shutting down, since craigslist has killed their income sources. So if we’re losing local journalism and the high-expense war reporting, where does that leave us?

One question I’d love to have the answer to is where the big expenses lie. Is it the Baghdad Bureau, or the printing press? If it is true that the NYT spends so much on printing and delivery that it can afford to send all of its paying subscribers a free kindle, does having a web-only presence allow it to continue paying for the high-quality journalism without paying for print media? A read-through of their quarterly report might help with that—-let me know if you know about this!

[update: put quotes around the parts I quoted from they Shirky piece]

Driving in India

The key thing to remember about driving in India is that it is different, not worse than driving in the US. It also works, which the US’s system would not at the scale of transport that India’s population requires. Here are some basic rules of life when driving in India.

Honking

If you do not honk in India, there are two possible reasons:

you are walking, or
you have a broken horn.

I even started honking at cars while walking in the road at some point, mostly in jest.

Here are some things that honking might mean:

I am passing you on the left
I am passing you on the right
You are getting too close to me
I am a car, you are a motorcycle
I am a car, you are a dog
Since you are a cow and are laying in the road resting, I will honk at you, but will make every attempt to pass you.
You are a truck and I am a car
You are going too slow (usually accompanied by flashing headlights)
You are making a U-turn or are a person crossing the street. Since I am driving my car in the correct direction on the highway, I have asserted myself and will not slow down, although I may change half of a lane.
I am going the wrong way down the highway
I am going around a turn, and don’t want to slow down, but can’t see if there is someone coming in the opposite direction
I haven’t honked in 1 minute
I haven’t heard a honk in 30 seconds.

If you don’t hear a honk in a while, something is wrong, and you should check your surroundings for danger. Horns on cars and motorcycles are standardized, but trucks and busses have horn sounds commensurate with their size. They all sound jolly, but you know to be scared when you hear the melody coming from a MACK-truck equivalent or coach bus. On a related note, trucks and vans play some pretty amazing songs when they go into reverse instead of the shrill beeping you hear in the US.

Lanes

Some roads have lane markers. Do not be confused, and do not ride in the lanes. Lane markers are for riding ON, not BETWEEN. If you’re not a truck but you’re staying in a lane, you’re doing it wrong. You should be darting between trucks more frequently, and you’re endangering everyone else who is doing it.

Motorcades

When a VIP passes you, their security guards will pass you by and wave you to the side of the road so that the VIP car can pass by. Do not be alarmed, as you’re not getting pulled over. Just wait for the caravan to pass, and then slam on the gas! AS soon as you are in the clear, keep as close to the caravan as possible. After all—-they are pushing people to the side of the road, so shouldn’t you be taking advantage of this to go 120km/h?

Motorcycles

By far, the most popular mode of transportation is the motorcycle. Saje tells us it costs 35000-50000 Rs ($700-$1000) for one. Most are sized way smaller than an American one, but look pretty stable and solid. The most common configuration is to have two people on the bike (often, women in Sarees sit sideways on the back, with the cloth dangerously close to the chain). Seeing three people (A family including a little child) or seeing one person riding it alone are both as likely. Seeing four people is not uncommon, and seeing two people with the person on the back carrying something that could snugly fit in the trunk of a small car is common. Random things I have seen the second person on a motorcycle holding have included a television or two 3-meter poles.

Motorcycles are the best way to get around. They don’t stop at stoplights (though few cars do, either), they don’t pay any tolls, and since they hold as many people as a car does, who needs more:)?

Helmets are not common, and if they are worn, only the driver wears it, not the child in front or in back of them. Protecting a small child means sandwiching them between parents.

Auto Rickshaws

These guys are only slightly less annoying in pestering you to take a ride than their pedal-powered equivalents. They have three wheels, and the power and stearing facilities of a motorcycle. Fully loaded, they have no pickup, which is why they try to never stop. Calling them a deathtrap seems unfair, since there are so many, and it’s so easy to pick one up for a several-kilometer journey. They vary in size, as we discovered when we got the smallest one and lost all feeling in our lower body while smushed in the back. Their turning radius is amazing though, and are good for pulling a semi-circle to drive the wrong way out of a traffic jam. I can’t imagine that they are more than a shell on top of a motorcycle, with all of the components exposed, and the driver sitting on top of the battery.

Cows

While dogs are more prevalent, cows (and bulls) are everywhere, including the middle of cities. They generally stay to the side of the road, but nothing stops them from walking in it, crossing it, or taking a nap in it. Drivers give them their respect, and living with them is quite nice. We saw one farm with cows on it, but otherwise, all cows we

saw were on the road, train platform, or anywhere where you might fancy eating some garbage. In cities, countrysides, and popular attractions, the cows would flourish. Interesting note: when cows do walk in the street (as in, walk in traffic, not just crossing two lanes of it), they seem to go with traffic, not against it—-even the cows have some decency when it comes to traffic rules.

Funny story about cows: While driving in a busy rotary, a cow was blocking our way. After some honking and waiting, Saje hit the gas a bit, and the minivan bumped the cow in it’s back half. It blinked quietly, fishtailed a bit, looked forward, and walked away, minding its own bovinely business.

Bicycles

Bikes are quite common, but these aren’t the Walmart Huffys or sweet roadbikes you see in the US. The closest thing I can compare it to is a commuter or hybrid bike, but without the logos or paints, with more rust, and with none of the parts matching. There is no bike lane, so bikes tend to drive where a lane would be if it was painted on the road (if the road had asphalt). They never wear a helmet, but ride slower and less aggresively than bikers go in Boston, so perhaps it’s for the better. Like motorcycles, you won’t be surprised to find two people on a bike, but three is less common. It’s also not uncommon to see these on some highways, along with pedestrians.

Trucks

These guys are narrower and shorter than their American breatheren, but carry way more. Rather than carry shipping containers, they resemble trucks that carry dirt in the US, but with stuff piled way higher. To secure those items, a tarp is loosely tied to the top, giving the illusion of security. I have seen MACK-like trucks carrying loose rocks piled higher than the top of the open container, with nothing holding them down except densepacking.

They drive slower than in America, but you must still watch for them. Many are painted in bright yellows, reds, and greens for decoration, and almost all of them say “Use Horn”and “Use dipper at night”, requesting that you utilize headlights.

Many truck cabins are pimped out, with everything from streamers, fuzzy things, cloves, and shrines adorning the inside. This makes passing them a whole lot more fun. The music they play is also to die for!

Mumbai and home

We arrived at the Victoria Terminus (VT is the British name for it, but it’s been renamed to CST since). This train station is supposed to be the architectural Taj Mahal of the British empire, but I can hardly compare the two. It was 6 am when we got off the train, and it was still dark out, so we waited to leave the station and dropped our big backpacks off at a holding area for the day.
Before leaving the station, a woman walked up to us and spoke to us in perfect English. We semi-consciously ignored her and walked away, as is the custom on your 18th day of being in India—-generally, you ignore anyone that makes a remark at you since they are just trying to sell you something, etc. I guess she was used to this, so she explained that she worked for the government as a tour guide (still dubious), and wanted to give us advice on where to go. She also said she has seen way more hollywood stars than us (her list was definitely longer than mine:)), and explained that she was born in England and moved later in life. She gave us some advice, and then we parted ways.
Before we left VT, I had a horrible need to go to the bathroom. Luckily, we were prepared with toilet paper and all. I walked into the (for pay) bathroom, and the guy taking money at the door took the toiler paper away and said “no paper.” I walked out, but Meredith pointed out I’d have no other option that day. So I walked back, and he was frisking for toilet paper, so I went in with none. I’ll save you the details, but you can read about how one goes to the bathroom in India elsewhere, and you can fill in the details of how on my last day in India, I became an inaugural member of the Indian population. While I was mentally preparing myself for the experience, I stood in a line of 20 people for two stalls. I had time to reflect, and was assisted by the man cleaning the urinals (which we were standing next to). He was soaked from head to toe, and wearing flip-flops. Cleaning meant walking from urinal to urinal with a hose, spraying EVERYTHING, an scraping the walls with a hay broom. Needless to say, the result was like being in the front row of a Gallagher show, and I became one of the dirtiest people in India. Using the toilet signs you up for someone smashing on the door if you’ve been in there for longer than a minute. Such is life.
After running out of the bathroom and bathing in baby wipes and hand sanitizer, we were on our way! We briefly got lost in the wrong direction looking for a cool market and ended up in a market with hundreds of chickens and their feathers. Breakfast at McDonalds because nothing else was open and I was going to fall over from low blood sugar. Of course their selection was waaaaay different than anything you’d have in the US for breakfast. Then we began a walking tour of Mumbai.
We made our way to a large green open area in between the high court and Mumbai university, where there were 5 or so simultaneous cricket matches going on. On every corner of the field, and throughout the city, there were vendors selling crushed sugar cane juice that looked amazing, but was served in a glass cup that was not washed after its last use. Being chicken, we avoided the drink even though it looked amazing.
Heading down a different street, we saw a neat church, a library run by the Asiatic society, and a way more green park than the last one.
We then kept walking down a major street, a block of which had Dildo vendors displaying their wares. Really? Dildos?
We turned off onto a smaller street to see the Keneseth Eliyahoo Synagogue. The guards at the door, when asked if we could go in, said “Are you Jewish?” After affirming, they let us in. The place was very pretty, painted sky-blue even on the inside, and having some of the most ornate designs I have seen in a synagogue. We bought a book on Jews in India, got a view from the top pews, and headed back into the street (no pictures, sorry!). On the way out, we took note that Madonna had been there two years ago. Heh.
We were hungry and thirsty (close to delirium because we were a bit irresponsible in walking around, and it was RIDICULOUSLY hot), so we bought some suspicious water at a street vendor. Your last day somewhere makes you feel invincible I guess.
We made our way to Gateway of India, a big structure that looks like a gate that is open to the Arabian sea/Indian Ocean. It was touristy, with mostly Indians posing for pictures. We then turned around from looking at the ocean to see the Taj Hotel, the site of the terrorist attacks on 11/26/2008 (known in India as 26/11). The hotel looked like a palace, and looks like it’s in great shape. There are signs along the entrance floor telling people that they are “Working to restore a symbol of Mumbai’s enduring spirit & dignity,” and I’m sure that once they do, the hotel will be regal as ever.
We went to a restaurant for lunch, and I had the first thali which seemed like what I would consider a Thali. It was very tasty, and only made taster by the fact that I ate with my hands. I hope my friends at home allow me to do the same when I am out with them.
We made a long walk back, essentially seeing the same sites we saw earlier. Stopping by a bookstore, we saw, front and center, a book on how to fix several brands of Nokia phone. I had heard about this repair-everything culture before, but it never struck me how prevalent it was. The book had circuit diagrams and listed every possible problem you could have with a phone and how to fix it.
We had some time to kill, but not enough to see the beach, so we went back into the McDonalds near Victoria Terminus. We shared an ice cream cone, and sat around watching high-schoolers do homework and flirt. Some car had a horrendously loud sound system, and one of the techno songs they played was the Venga Boys song we had heard on our first night. It was a relaxing sendoff in a bustling city.
We got a taxi at the station after getting our bags. The ride took more than an hour in intense traffic, but we were leaving at 6 pm and the plane took off at 1 am, so no biggie. We drove through some slums (55% of Mumbai lives in such slums). Essentially, these are corrugated tin-walled living spaces stacked on top of each other so you have to use a ladder to get from one family’s floor to the next. No one looked terribly sad (but this is from the highway. I hesitate to use the word slum, because while the word slum implies how it looks, the slum is far from what you would expect economically. Despite the surroundings, people have set up a functioning economy, where there are home renters and owners, people work at various locations, and the economy of the slum exports $550 billion worth of products per year to the rest of the world. Not somewhere you’d like to live, but not exactly the complete image you might paint for yourself.
As we drove past the slum, we saw an astounding amount of kites in the sky. If you have read the “Kite Runner,” you’d have the explanation of how it’s done (at least in a fiction book based in Afghanistan). From the book, you glue glass shards to the string, fly the kite, and try to drive your kite’s string into another kite’s so that you can cut it down. The kite running part comes from the point at which you cut the kite—-your partner-in-crime, once the opponent’s kite is cut, makes a mad dash toward where the kite might fall, and races half of the other children in the neighborhood to get the prize. I’m not sure that part actually happens given that running around to random places might be unacceptable, but it certainly sounds poetic. We had seen this throughout India, but as a final vision of the place during the waning hour of sunlight, it was quite exhilarating to see tens of kites flying high over the neighborhoods of Mumbai.
Equally exhilarating was the way in which we ate dinner at the airport. After trying to reason with the driver to know how much markup he charges westerners (“come on, I’ll pay you what you wanted, but I’m leaving India and want to know how much you overcharge westerners” didn’t work), we tried to get past the security check at the entrance to the airport. We arrived at 7 pm for a 1 am flight, so the guard told us that there is no food in the terminal, and that before going in, we should “go upstairs” and get a bite to eat. We thought it was strange that there was no food inside, but took his advice. We went upstairs, where we explained to the guard with a massive gun why we were up there, to which he kept responding “What’s wrong?” until he let us go. We walked for a good 2 minutes in a long corridor that seemed to not be designed for public consumption. Finally, we were pointed in the right direction by someone who looked very confused at out presence. Upon walking in, we were welcomed by about 100 employees of the airlines turning around and staring at us, and saw the banner at the back of the room, which designated this the “employee canteen.” We were about to turn around when someone ran toward us and told us to sit down and eat anyway. What a wonderful man! For ~$1 each, we got 2 chai, 1 uttapam (a dosa infused with potato), a cheese sandwich (did I mention Meredith was stuffed with Indian food?), Pani Puri (deep-fried bread stuffed with potato and tamarind, and drizzled with tamarind juice and yogurt), wada (like a samosa, but with fluffier breading), and an atmosphere that made you feel at peace with yourself as you waited. Once we were ignored by our neighbors, we felt at home and enjoyed ourselves. Upon returning to the airport to enter the terminal, we saw that the guard had changed, and that the place was filled with places to eat. I’m happy he tricked us—-what a great final dinner in India!
At the airport, we changed clothes and took baby-wipe showers. We went through two security scans, and spent/exchanged the rest of our money (the exchange system enrages me, but I won’t get into that I guess).
The place ride home was delightful—-at around 2 am India time, we were served dinner, which everyone on the plane except for me refused. I then joined everyone else in passing out for a few hours, and then woke up to watch some shows on my seat’s TV. Our neighbor ended up being not only talkative but immensely interesting. He’s a Christian from Goa, and speaks perfect English. He works as an engineer in Goa, and travels to New Hampshire to share knowledge with the sister plant several times per year. We spoke about the massive cultural differences between Hindi-speaking India and much of the south (from his perspective). We also spoke about peace (India/Pakistan, Israel/Palestine—-two different perspectives on different places). He was interested in Meredith’s engineering background, and what I did at school. I’m happy he was there to share some final words with—-we’re now in e-mail contact, and he has an open invitation to stop by Boston the next time he is in town! One interesting thing he mentioned was reverse culture-shock. Sure, you’re shocked by the other’s culture when you first land, but when you return to your country, you’re shocked again by how different everything is, as if you never saw it before.
That was our experience upon arriving in the US. Culture shock was everywhere, in a strange way. I felt uncomfortable crossing the street without cars beeping to acknowledge me. We were stricken by the whiteness of the walls in our apartment, and the cleanliness of pretty much everything relative to what we were accustomed. For some reason, since returning, I no longer drink filtered water, and I think the reason is that I am trying to convince myself it’s a nicety that doesn’t matter, and I thus can’t justify anymore. Weird small things, but they add up.

Goa

Landing in Goa after our goodbye to A.J. (1-hour flight), we made our way into the hot hot sun. Goa is something close to paradise if you love heat, relaxation, and Bob Marley. Patrick (our bed+breakfast owner’s son) picked us up in a car playing Casey Kasem top 20 list of American songs. It was refreshing to drive along beautiful beaches and hear The Killers et al.
Our bed and breakfast was called Martha’s (after the proprietor), and the sign in front advertises the restaurant’s amazing waffles. The place was the smallest, quietest, and dantiest one we stayed at, and we quite enjoyed ourselves. Martha and her family ran the bed+breakfast, and so her two Daughters handled the restaurant and checkins, her son took care of taxi services and motorcycle rentals (I think?), and her husband helped with all sorts of odds-and-ends. They were warm people, and really tried to make us feel at home (next to their farm with pigs running around!). They said they would remember us if we ever contacted them again, since Meredith is such a unique name. We’ll see:).
While Goa is one of the smallest Indian states, the beaches and spice plantations, which, aside from Panaji, its capital, are the only sights, are along a four-hour stretch of land.
Almost everyone travels by motorcycle or scooter here, and not renting one put us at a great disadvantage, since taxis couldn’t be haggled with (they were called by the bed and breakfast, and knew we had no way to get to a place where we were in a position to haggle). Meredith and I were scaredycats, so we took the hit anyway.
The beaches were amazing. I’ve never been to the bahamas, but I imagine that’s why people go to these places. Meredith and I burn easily, and so we avoided them for the most part except for to take pictures and watch the sun set.
Since the Bahamas are so close to the US, it makes more sense for Americans to go there. As such, we saw three other Americans (in one group), and the rest of the folks where European. Mostly German and British folks.
Since the place was more of a resort-like area, and the primary customers where “rich” foreigners, the food was vastly more expensive than it was elsewhere. This meant we spent $8-10 equivalent for two of us on a dinner or lunch. The meals were great though - we had a rice with fish curry dish, Xacuti, fish tandoori, all of which are in some way specialities of Goa.
Everywhere else in India, it’s not uncommon to see a shrine set up by the side of the road by some religious group with flowers hanging around the diety. Since Goa has way more Christians, it was common to see a shrine to a crucifix by the side of the road, also with flowers hanging around it.
Side note: the reason Goa has so many Christians is that until 1961 (formally before that), it was a trading colony of the Portuguese. Many more people spoke perfect English (I don’t know how one follows the other), and everyone we spoke with that spoke excellent english was Christian.
Since we didn’t spend much time on the beach, we took a taxi into Panaji, the capital. We tried a walking tour around the city, and got lost a bit, but it was all through fun places to see. I had coconut juice (we really let loose here—-we were no longer concerned for our health for some reason:)) out of a coconut. Coconuts are way more green and smooth on the outside, and the juice is very tart. The vendor charged 20 rupees (40 cents) for one, and I got the pleasure of seeing him hack the top off of it to make a sipping hole with a machette. Our walk brought us to many beautiful churches (some of the cleanest white buildings we saw in India). We also walked through smaller neighborhoods, crossed a river a few times, and found a humungous temple dedicated to the monkey god Hanuman. The building was either salmon pink or orange depending on the paint used (and the colorblindness). Meredith pointed out some flower-shaped relief sculptures at the entrance that had half of their petals broken off. The solution in this case was to paint the petals back on, meaning that half of the flower was a sculpture on a wall, and half of it was paint on the wall. Really interesting way to fix things that need to “just work.”
To save money, we found an autorickshaw to take us back to our hotel. He was waaaay cheaper than the taxi (I’m embarrassed to say how much the taxi was, but the ballpark was $10, whereas the autorickshaw was ~$4). The ride each was to Anjuna beach (our hotel) was 30 highway minutes, so perhaps the cost was justified in petrol.
Upon heading back to our hotel, we went down to the beach to see the sun set—-beautiful! We then walked back and wondered why everything was so dark (made it hard to find restaurants). Finally, we walked into a completely pitch-black one. They explained that the power was out everywhere, but that they could still prepare dinner for us. We decided to give it a shot. This place was classy, and we were not dressed our part, but we were the only ones there, so no biggie. They made veggie Lasagna and veggie Musaka, both of which were Goan takes on the meals (unexpected vegetables, cumin seeds and the works). Afterward, we asked them for a candle (I had a flashlight, but we wanted to be protected from motorcycles in both directions walking home), and they fashioned us a candle holder our of a soda can. Rock!
Since most of the visitors we saw were European (either college-age or burnt-out hippies in better shape than us), our Goan beaches catered to their likings: beach-front raves, and yogurt. The raves were an all-night thing (even though being caught with drugs can land you in jail for a long time), and even featured fireworks (how?!?!). A German bakery we visited (I swear it was hard to find a south Indian restaurant around here) let you sit on the ground, drink various expensive teas, and eat mounds of yogurt and fruit. While I won’t say Goa made me more enlightened, it was fun to witness the place.
The train to Mumbai was the Konkan Kanya express, which goes along the Konkan coastline. This is supposed to be the most beautiful coastline in the world, but the train (in both directions) for the most part travels at night, so you don’t actually get to see anything. We had some great food on the train for 80 cents each. The first dish was a standard vegetarian dish with dal, rice, some vegetables, chapati, and pickle (mango in brine). The second and more interesting dish was veg manchurian (every vegetarian dish is called ‘veg’), which was some amazing sauce and hardly recognizable balls of something which I would say is doughy, but I can’t be sure. It ranks up there as one of the best dishes I had on the trip, but I honestly can’t put my finger on what it was.

Kolkata

Our train arrived on time, which was refreshing. When we got off and went to the prepaid taxi booth, we were hassled pretty heavily by the drivers to go with them, which is usual. By this point, we had gotten used to it and ignored them, but we’d break every once in a while and start repeating “no!” at them. Kolkata is the intellectual capital of India, and it shows; a smiling taxi driver enjoyed A.J.’s chorus of “no!,” but decided to give him a quick education. As he turned to walk away, the driver corrected A.J., explaining that “It’s pronounced nay!” Thanks.
No other city had as many yellow taxis as Kolkata, and we later found out that this was due to a recent supreme court decision to ban autorickshaws due to emissions. We took a taxi to the hotel, dropped our stuff off after waking up the poor hotel employees, and took another taxi to Chowringhee street, which had the majority of the tourist attractions we would be visiting.
We made our way down a few streets by foot looking for a cheap eat recommended for breakfast in our Lonely Planet book. The food was amazing and cheap, and we even had the pleasure to have the largest beetle I have ever seen bless the floor with its presence. I learned that of the three of us, I am the only one disturbed by insects larger than my pinky.
Making our way back to Chowringhee street, we entered the Indian museum, the largest museum in India, and one of the oldest in the world. Exhibits included animal skeletons, rocks, paintings, insect and animal replicas, and an awesome room containing dioramas of the major cultures of India.
We walked away from the main drag, which was a tourist trap larger than any one we had seen before. Along our walk, we were approached by several men that talked us up convincing us to go to their store a few blocks away. One even told us that we looked like we were from California (LA specifically), and I didn’t have the heart to tell him that’s not a compliment. At a major intersection, A.J. darted across and left us in the dust. Meredith and I tried to stick together, but she screamed “BUS!” at some point and ran back to the sidewalk. I was stuck between a bus and a trolley, and had a bus driving at me. Luckily, I was in my predicament with a natural problem solver, who slapped the side of the bus blocking us as we walked around it, dodging other traffic until we made our way across. Meredith had an easier time, luckily.
Heading to a quieter part of town, we ate at an indo-chinese restaurant that wasn’t as good as one I tried in New York, but was still a good mix of Indian and Chinese foods.
After lunch, we tried Bengali desserts—-Roshgulla (soft cheese balls in a sugary syrup, less floury than gulab jamun) and Sandesh (finely ground cheese and seeds forming a dense Halva-like texture). A child tried to grab my dessert box, but I showed him it was empty, which seemed to be good enough for him to leave me alone.
We then walked back toward the middle of town and walked for a while along the park. We made our way to the Victoria memorial, which was closed by the time we made it. We walked around the park for a while, and stopped by an open-air concert with an amazing female vocalist. We then continued on, looking for the english-language water/light show as night fell. We were told that for some reason it wasn’t happening tonight, but as we left disappointed, we saw the beginning of a 5-minute version of the show. It was nice to sit down, and while we didn’t get a history of Kolkata, it was a good break.
Arguing for lower rates from taxi drivers, we went down a line of taxis asking each one for a better rate until we were happy, and headed back for the hotel. It was too late to get a good fish dish, which disappointed me, since Debmalya told me the fish would be amazing. We got some tea near the hotel instead, and enjoyed our assortment of crackers back at the hotel. We made final packing plans, as the next morning we would get in a taxi and head off to the airport.
We woke up crazy early, and got to the Kolkata airport with plenty of time to spare. Security is about as dumb as it gets—-you first scan your bag, which is wrapped with easy-to-subvert plastic ties, and then carry the bag to the ticket coutner, where it is (presumably) not scanned again. At least the US has something right—-scan the bag after the person carrying it no longer has possesion of it. Oh well.
Flew to Mumbai (2 hours) and grabbed our bags at the super-futuristic Mumbai domestic airport. It’s actually not quite so super-futuristic, but Thomas Freidman tries to convince everyone it is. Good for him—-it’s clear he’s only ever been to Mumbai and Bangalore:)
We said our sad goodbyes to A.J., who told me I smelled as I hugged him goodbye. Turns out he had a pretty annoying experience coming back: first he almost got duped by a taxi driver while trying to see Mumbai proper, only to turn around and head back to the airport, and then got to Delhi for his flight to JFK early only to hear the the Delhi flight left 1.5 hours BEFORE it was supposed to take off to avoid fog. After much arguing, he got on a different flight home, which took him through Belguim. Poor guy spent over 48 hours in some kind of transport state before getting home.

Darjeeling

We had an awesome night in the hotel in Siliguri, which featured a dinner fit for the elite, though one we could hardly characterize as Indian. A.J.’s dish was the most amazing one, featuring a sizzler (it sizzled loudly as it was brought out) of vegetables and paneer served in a cabbage. The sauce was amazing, and the vegetables rocked!
After breakfast, we took a jeep for a two-hour trip to Darjeeling. We were originally going to arrive in Darjeeling the night before and take the toy train back to Siliguri, but since we were delayed a day, we opted for the jeep. The jeep followed the same path as the train, but took two hours instead of eight, so we felt vindicated.
The path to Darjeeling was beautiful - it was the only all-green, all-mountain experience we had in India. There were tea plantations everywhere along the way (Darjeeling is known for its teas, as is Asam, a state to the east). As we approached Darjeeling, it became clear that the big-city atmosphere we became accustomed to everywhere else was gone, which was relaxing in its own right.
We got a quick (three-hour) tour of Darjeeling, which featured a visit to a Buddhist monastery, a ride to tiger hill (the highest point in the area), and a trip to the War Memorial Batista Loop. The memorial, dedicated to Indian Gorkha soldiers from Darjeeling who had died at war since independence, gave us a beautiful view of the surrounding peaks and valleys. Tiger hill is supposed to be visited at 4 am for sunset, when it is clear enough to see Mount Everest and other fun sights. The fog at noon blocked all visibility, so we took pictures with where Everest would have been, and looked equally happy.
Upon finishing the tour, we wanted to get some momo (dumplings, tibetan-style) and Darjeeling tea. We asked someone where to go, and they suggested some chicken/pork places. Upon further prodding for a vegetarian place, he led us down a small alley to a no-menu joint that was hardly more than a kitchen with tables, and definitely didn’t see foreigners. We ordered soup, 8 momos each, and tea, all of which were amazing. I initially thought I was jipped a momo, but after we finished, our waitress plopped another on my plate, reminding me that I only received 7 to start. The bill came to 39 Rs (80 cents) for the three of us. Amazed by the low price, we decided to leave a hefty tip (something that seemed normal in the bigger cities, and still put us at around 40 cents apiece). The waitress ran after us telling us we made a mistake, but we happily told her to keep the rest. Hopefully we didn’t offend anyone.
On the way back, our driver stopped at a tea factory (that produces some teas for Tazo, the Starbucks people), where we learned about tea production in a very dark and scary set of rooms, and then bought some souvenier teas.
Our driver was pretty cool, and explained lots of things to us. The most interesting involved the independence movement in Darjeeling. From what I could understand, residents feel that because many citizens are ethnically different from the rest of West Bengal (we didn’t see many people from tibet or nepal in Kolkata), they deserve independence. I couldn’t get him to go farther than that, but it was interesting nonetheless.
Meredith felt sick on the car ride to the train station. The train arrived on time, and once we boarded, it was pretty clear that Meredith had whatever I had gotten over. Instead of putting her through several days of discomfort, we put her on antibiotics as well, and she woke up in Kolkata several hours later feeling like a champ!

← Earlier Posts Page 1 of 3