gibson.yang's blog

Tokyo Tyrant is a high concurrency network interface to the underlying database (Tokyo Cabinet).

It has been widely adopted in many web-related applications as backend data store.

This post will introduce Tokyo Tyrant's backup and restore capabilities.

Backup

figure 1.

The following example shows how to hot backup a ttserver, and recover a ttserver using the backup database as illustrated in figure 1.

 - first, start a ttserver.

ttserver /tmp/ex1.tch

 - on another terminal, write some data into it.

tcrmgr put localhost "foo" "bar"

 - check if we have a record (key:"foo", value:"bar").

tcrmgr get localhost "foo"

 - backup the database.

tcrmgr copy localhost /tmp/backup.tch

 - let's simulate the ttserver crashed by press crtl-c in the first terminal.

 - and database file is corrupted.

rm /tmp/ex1.tch

 - to recover the ttserver from previous backup, simply copy the database file and restart ttserver.

cp /tmp/backup.tch /tmp/ex1.tch

ttserver /tmp/ex1.tch

 - check if the data has been recovered

tcrmgr get localhost "foo"

Data loss can be prevented by scheduling periodically backup (once a day or a few minutes, depends on how frequently the writing operations are). But we still gonna lose all the modifications between the last and the next backups. Tokyo Tyrant also provide an approach to restore those modifications.

Restore

figure 2.

The following example shows how to restore the modifications from update log as illustrated in figure 2.

 - create a directory to hold update log.

mkdir /tmp/ulog

 - start a ttserver with update log enabled.

ttserver -ulog /tmp/ulog /tmp/ex2.tch

 - on another terminal, put some data into it.

tcrmgr put localhost "foo2" "bar2"

 - check if we have a record (key:"foo2", value:"bar2")

tcrmgr get localhost "foo2"

 - let's simulate the ttserver crashed by press crtl-c in the first terminal.

 - and the database file is corrupted.

rm /tmp/ex2.tch

 - instead of recovering the ttserver from backup, we can restore the database by replaying the update log.

 - backup the update log if needed.

mv /tmp/ulog /tmp/ulog-back

 - restart ttserver.

ttserver /tmp/ex2.tch

 - restore from update log.

tcrmgr restore localhost /tmp/ulog-back

 - check if the data has been recovered.

tcrmgr get localhost "foo2"

This newly announced project is featured in New York Times on May 12 - "Four Nerds and a Cry to Arms Against Facebook". First line of the article says "How angry is the world at Facebook for devouring every morsel of personal information we are willing to feed it?".

Almost all social network services presenting today are centralized, such as Facebook, Twitter, Orkut etc. we fill out personal information to register as an user, hand over messages via their servers to communicate with our friends. In the mean while, what we are giving up is all of our own privacy. That may increase data leakage and we have to be more cautious about what we are posting on these social networks.

A few months back, four geeky college students of NYU (Mr. Salzberg and Mr. Grippi are Raphael Sofaer, 19, and Ilya Zhitomirskiy, 20), decided to build a social network that wouldn’t force people to surrender their privacy to a big business in exchange for convenient access to their sites. They have called their project Diaspora and intend to distribute the software free, and to make the code openly available so that other programmers can build on it.

The Diaspora group was inspired to begin their project after hearing a talk about "internet privacy" by Eben Moglen, a law professor at Columbia University. As more and more of our lives and identities become digitized, Moglen explains, the convenience of putting all of our information in the hands of companies on “the cloud” is training us to casually sacrifice our privacy and fragment our online identities. Why is there no good alternative to centralized services that, as Moglen pointed out, comes with "spying for free?”

“When you give up that data, you’re giving it up forever”

“In our real lives, we talk to each other, We don’t need to hand our messages to a hub."

"Our real social lives do not have central managers, and our virtual lives do not need them." — said by Diaspora group.

The project is described as a "network that allows everyone to install their own “seed” — i.e. a personal web server with a user’s photos, videos and everything else — within the larger network. That seed would be fully owned and controlled by the user, so the user could share anything and still maintain ownership over it".

It would take three or four months to write the code, and they would need a few thousand dollars each to live on. They gave themselves 39 days to raise $10,000, using an online site, Kickstarter, that helps creative people find support. They announced their project on April 24. They reached their $10,000 goal in 12 days, and the money continues to come in: as of today (May 24), they had raised over $180,000

Not bad for a financial start to turning an envisioned new network into reality. It is far too soon to tell whether Diaspora will replace Facebook and become the next top social networking website, however due to the ripe timing and tremendous amount of support, it might just have a shot.