Back in the days of DOS, before steam, in the long-long ago, there were tools like SoftICE and IDA and W32Dasm that would allow you to hook into your operating system or running executables and modify them in perverted ways. Today, I’m going to show you a little trick from my game cracking days for unlimited health in the game Terraria, which I tend to play a lot when I don’t have access to my gaming machine.

Memory searching

Every game has variables that store information: your ammo, your health, how large your penis is, etc. Each of these variables generally has a dedicated position in memory where it’s stored. If you can find the location of this value, you can modify it to your liking. But how to you find it? This is the process of memory searching.

The tool we’re going to use is called TSearch. I don’t think it has an official site anymore (assuming it ever did), but you can get it off this completely legit Angelfire website (direct link). While I can’t guarantee the executable is free of pathogens of the digital persuasion, I can say that I’ve scanned it with several tools and come up blank. UPDATE: There are probably better tools you can use, for instance Cheat Engine, an open source program in the same vein as TSearch. I Chose TSearch because that’s what I used in the old days, and had a copy laying around on my machine.

The way memory searching works is you pick a value you want to modify. We’re going to do health. So open start Terraria, open your favorite world, and let’s go to town. Once Terraria is loaded, open TSearch. On the top-left, you’ll see a button called “Open process.” Click it and select Terraria.

Now that the process is open, TSearch has access to Terraria’s memory, including your health value >=]. We’ll start our search for the location soon, but first we need a starting point.

The next step is to hurt yourself in Terraria. Yes, we’re going to to be playing the part of the masochist (but only for a little while). So jump off that cliff, dip your toe in lava, tango with a skeleton, whatever it takes…you want your health value to not be in its default position.

We’re going to search for that health value in Tsearch and as a starting point to try and find its exact location in memory.

Once we have lowered our health, open Tsearch. Make note of the value your health is in Terraria (and also make sure the game is paused).

Open TSearch and click on the little magnifying glass icon under the “Open process” button (the one furthest to the left). It will bring up a dialog that allows us to search for locations in memory based on value. We’re going to pull down the menu and select “range.” We use “range” instead of “exact value” because the health value regenerates, meaning at any given point it might be between two integers.

Let’s say our health is at 268 (like the image to the right). For value one, we enter 267, and value two 269. Then select Type: “2-byte.” This means we’re searching for a 2-byte value between 267 and 269. Hit “Ok.”

It’s going to find probably tens of thousands of values. Don’t worry, we can narrow these down. Open Terraria again and let your health regenerate (or hurt yourself again, being careful not to kill yourself). Once your health changes by a few points, open TSearch again.

We’re going to search again, but search within our current results. Click the magnifying glass directly to the right of the first one (it has “…” on it). This will open the same dialog before. Do a ranged search. Let’s say our new health value is 259, we’re going to search between 258 and 260. Run the search.

If you’re lucky, you’ll get only one result. If you’re unlucky and you get more than one search result, repeat the above filtering step one more time (change the health value, filter withing the current results, rinse & repeat until only one value remains). Sometimes you’ll have to do many searches in a row to get the results you want.

Once you have your one search result, click on that entry in the search box and then click the green plus icon. It should add it to the list on the right. Double click under the “Description” field and name the value “health.” On the right, you’ll notice the value 259. Change it to 400. The click the box next to the value’s name (it should turn blue). You just froze your health at 400.

Go back into the game and hurt yourself. You’ll notice that your health goes down, but immediately bounces back after about half a second. Ladies and gentlemen, enjoy your infinite health =].

Notes

Hopefully this can answer some more questions about memory searching.

• This method only works while TSearch is open. Once you close it, the game will start tracking health again. If you want your hacks to persist, it’s a whole nother set of tools and processes, worthy of at least a few more posts. Keep in mind that if you use Steam, it will probably overwrite any changes you make to the exe. Such is life.
• Memory searching works for values besides health. Be creative! You can get unlimited ammo, unlimited money, etc.
• In Terraria, using this method for money can be tricky. If you move money from one of the money slots, it will change memory locations and you’ll lose your hack.
• This method works for other games as well. I chose Terraria because it’s the only game I currently use it on.

Have fun!

UPDATE: Since RethinkDB 1.4, this post is pretty irrelevant. You can now just do:

./configure --allow-fetch
make ALLOW_WARNINGS=1

This will build RethinkDB without a hitch.

UPDATE: Check out Samuel Hughes’ comment on compiling in Slack, which may make some of the below process simpler. Specifically, the section about editing the Makefile to not use static libraries (apparently you can pass RECOMMEND_STATIC=0 to the make process to do this instead).

So I’ve had a theoretical boner for RethinkDB ever since reading about it. I decided to try and give it a go, but have had problems compiling. I’m going to try and give an overview of how to get it running. These instructions are aimed at people who don’t have a linux with the targeted packaging systems that RethinkDB currently supports (in other words, you’re stuck with compiling it yourself). The build process is slightly annoying (which is why I’m writing this guide). Slava from the RethinkDB team told me that they are working on a new build system, so hopefully we’ll soon be able to just to a make.

Installing V8

See the instructions for building and installing V8. They are pretty simple, but I believe you need Python (since they use GYP for the build):

cd v8
make dependencies
make native
# do a manual installation
sudo mkdir -p /usr/local/v8/include /usr/local/v8/lib
sudo cp include/* /usr/local/v8/include
sudo cp out/native/lib.target/libv8.so /usr/local/v8/lib

Done (yeah, I know, my syntax highlighting is annoying).

Building RethinkDB

This was a bit trickier, but hey I’ve compiled alpha versions of Compiz on top of Slack before, this should be a cakewalk. It took some Makefile tweaking to get it running, so here’s how I did it. Note that you’ll need Python >= 2.7 to do the full make process (or else you’ll get “AttributeError: ‘module’ object has no attribute ‘check_output’”). Slack 13.1 comes with 2.6.x so I had to compile it. Guess I need to upgrade soon. So grab the latest source:

git clone git://github.com/rethinkdb/rethinkdb.git
cd rethinkdb

First things first, src/Makefile adds the option -Werror to the build. This is great, but causes the build to fail when it includes v8.h since there are unused variables. So for now, we’ll have to trust them that there are no other warnings/errors and remove this from the Makefile. So open src/Makefile in your favorite editor and change:

RT_CXXFLAGS+=-Wall -Wextra -Werror -Wnon-virtual-dtor

to

RT_CXXFLAGS+=-Wall -Wextra -Wnon-virtual-dtor

Also the Makefile tries to use static versions of some of the boost libs, but I only have dynamic versions on my system. So let’s comment out that line. Find this and comment it out (UPDATE: per Samuel Hughes’ comment below, you can skip this step and pass RECOMMEND_STATIC=0 to the make commend instead of hacking up the Makefile):

STATIC_RECOMMENDS_INDIFFERENT:=boost_serialization boost_program_options

becomes

#STATIC_RECOMMENDS_INDIFFERENT:=boost_serialization boost_program_options

Now let’s write a “simple” build script that wraps the make process:

#!/bin/bash

BIN="`pwd -P`/support/usr/bin"
export PATH=$PATH:$BIN
export RT_CXXFLAGS="-I./include -I./src -I/usr/local/v8/include"
export RT_LDFLAGS="-L/usr/local/v8/lib ../support/usr/lib/libprotobuf.a ../support/usr/lib/libtcmalloc_minimal.a -lboost_program_options -lboost_serialization"
export STATIC_LIBRARY_PATHS=$RT_LDFLAGS
make \
        VERBOSE=1 \
        FETCH_INTERNAL_TOOLS=1 \
        RECOMMEND_STATIC=0 \
        STATIC_LIBRARY_PATHS="$RT_LDFLAGS"

Save it to compile.sh (or just c like I do because life is just too short to be typing “ompile.sh” all over the place) and run it.

That should do it! If all the post-build stuff (like installing js modules and crap) works fine, you should be able to start the db like so:

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/v8/lib/ ./build/release/rethinkdb

If that’s too cumbersome for you, you can either link /usr/local/v8/lib/libv8.so to /usr/local/lib or add /usr/local/v8/lib to your /etc/ld.so.conf file (and, of course, run ldconfig as root) and run rethinkdb freely, without the worries of library paths.

I recently installed FreeBSD 9.1-RELEASE on a VirtualBox VM to do some cl-async testing. I wanted to get Xorg running so I could edit code at a more “comfortable” resolution. I was able to get Xorg running fairly easily just by installing Xfce from /usr/ports.

However, upon starting Xorg, my keyboard mouse would not work. I tried many things: following the steps in the handbook, enabling/disabling hald, reconfiguring Xorg, etc. No luck. My Xorg.0.log was telling me that it couldn’t load the kdb/mouse drivers. After snooping around some forums, I found the solution:

• Install the x11-drivers/xf86-input-keyboard port
• Install the x11-drivers/xf86-input-mouse port

After doing this, all was right with the world. Just to clarify, I am using dbus/hald and more or less using the default configuration that Xorg -configure gave me.

A while ago, I released cl-async, a library for non-blocking programming in Common Lisp. I’ve been updating it a lot over the past month or two to add features and fix bugs, and it’s really coming along.

My goal for this project is to create a general-purpose library for asynchronous programming in lisp. I think I have achieved this. With the finishing of the futures implementation, not only is the library stable, but there is now a platform to build drivers on top of. This will be my next focal point over the next few months.

There are a few reasons I decided to build something new. Here’s an overview of the non-blocking libraries I could find:

• IOLib – An IO library for lisp that has a built-in event loop, only works on *nix.
• Hinge – General purpose, non-blocking library. Only works on *nix, requires libev and ZeroMQ.
• Conserv – A nice layer on top of IOLib (so once again, only runs on *nix). Includes TCP client/server and HTTP server implementations. Very nice.
• teepeedee2 – A non-blocking, performant HTTP server written on top of IOLib.

I created cl-async because of all the available libraries, they are either non-portable, not general enough, have too many dependencies, or a combination of all three. I wanted a library that worked on Linux and Windows. I wanted a portable base to start from, and I also wanted tools to help make drivers.

Keeping all this in mind, I created bindings for libevent2 and built cl-async on top of them. There were many good reasons for choosing libevent2 over other libraries, such as libev and libuv (the backend for Node.js). Libuv would have been my first choice because it supports IOCP in Windows (libevent does not), however wrapping it in CFFI was like getting a screaming toddler to see the logic behind your decision to put them to bed. It could have maybe happened if I’d written a compatibility layer in C, but I wanted to have a maximum of 1 (one) dependency. Libevent2 won. It’s fast, portable, easy to wrap in CFFI, and on top of that, has a lot of really nice features like an HTTP client/server, TCP buffering, DNS, etc etc etc. The list goes on. That means less programming for me.

Like I mentioned, my next goal is to build drivers. I’ve already built a few, but I don’t consider them stable enough to release yet. Drivers are the elephant in the room. Anybody can implement non-blocking IO for lisp, but the real challenge is converting everything that talks over TCP/HTTP to be async. If lisp supported coroutines, this would be trivial, but alas, we’re stuck with futures and the lovely syntax they afford.

I’m going to start with drivers I use every day: beanstalk, redis, cl-mongo, drakma, zs3, and cl-smtp. These are the packages we use at work in our queue processing system (now threaded, soon to be evented + threaded). Once a few of these are done, I’ll update the cl-async drivers page with best practices for building drivers (aka wrapping async into futures). Then I will take over the world.

Another goal I have is to build a real HTTP server on top of the bare http-server implementation provided by cl-async. This will include nice syntax around routing (allowing REST interfaces), static file serving, etc.

Cl-async is still a work in progress, but it’s starting to become stabilized (both in lack of bugs and the API itself), so check out the docs or the github project and give it a shot. All you need is a lisp and libevent =].

We’re building a queuing system for Musio written in common lisp. To be accurate, we already built a queuing system in common lisp, and I recently needed to add a worker to it that communicates with MongoDB via cl-mongo. Each worker spawns four worker threads, each thread grabbing jobs from beanstalkd via cl-beanstalk. During my testing, each worker was updating a Mongo collection with some values scraped from our site. However, after a few seconds of processing jobs, the worker threads begin to spit out USOCKET errors and eventually Clozure CL enters it’s debugger of death (ie, lisp’s version of a segfault). SBCL didn’t fare too much better, either.

The way cl-mongo’s connections work is that it has a global hash table that holds connections: cl-mongo::*mongo-registry*. When the threads are all running and communicating with MongoDB, they are using the same hash table without any inherent locking or synchronization. There are a few options to fix this. You can implement a connection pool that supports access from multiple threads (complicated), you can give each thread its own connection and force the each thread to use its connection when communicating, or you can take advantage of special variables in lisp (the easiest, simplest, and most elegant IMO). Let’s check out the last option.

Although it’s not in the CL spec, just about all implementations allow you to have global thread-local variables by using (defparameter) or (defvar), both of which create special variables (read: dynamic variables, as opposed to lexical). Luckily, cl-mongo uses defvar to create *mongo-registry*. This means in our worker, we can re-bind this variable above the top level loop using (let) and all subsequent calls to MongoDB will use our new thread-local version of *mongo-registry* instead of the global one that all the threads we’re bumping into each other using:

;; Main worker loop, using global *mongo-registry* (broken)
(defun start-worker ()
  (loop
    (let ((job (get-job)))
      (let ((results (process-job job)))
        ;; this uses the global registry. not good if running multiple threads.
        (with-mongo-connection (:db "musio")
          (db.save "scraped" results))))))

New version:

;; Replace *mongo-registry* above worker loop, creating a local version of the
;; registry for this thread.
(defun start-worker ()
  ;; setting to any value via let will re-create the variable as a local thread
  ;; variable. nil will do just fine.
  (let ((cl-mongo::*mongo-registry* nil))
    (loop
      (let ((job (get-job)))
        (let ((results (process-job job)))
          ;; with-mongo-connection now uses the local registry, which stops the
          ;; threads from touching each other.
          (with-mongo-connection (:db "musio")
            (db.save "scraped" results)))))))

BOOM everything works great after this change, and it was only a one line change. It may not be as efficient as connection pooling, but that’s a lot more prone to strange errors and synchronization issues than just segregating the connections from each other and calling it a day. One issue: *mongo-registry* is not exported by cl-mongo, which is why we access it via cl-mongo::*mongo-registry* (notice the double colon instead of single). This means in future versions, the variable name may change, breaking our above code. So, don’t update cl-mongo without testing. Not hard.

Hopefully this helps a few people out, let me know if you have better solutions to this issue!

I’ve been seeing a lot of posts on the webz lately about how we can fix email. I have to say, I think it’s a bit short-sighted.

People are saying it has outgrown it’s original usage, or it contains bad error messages, or it’s not smart about the messages received.

These are very smart people, with real observations. The problem is, their observations are misplaced.

What email is

Email is a distributed, asynchronous messaging protocol. It does this well. It does this very well. So well, I’m getting a boner thinking about it. You send a message and it either goes where it’s supposed to, or you get an error message back. That’s it, that’s email. It’s simple. It works.

There’s no company controlling all messages and imposing their will on the ecosystem as a whole. There’s no single point of failure. It’s beautifully distributed and functions near-perfectly.

The problem

So why does it suck so much? It doesn’t. It’s awesome. The problem is the way people view it. Most of the perceived suckiness comes from its simplicity. It doesn’t manage your TODOs. It doesn’t have built-in calendaring. It doesn’t give you oral pleasure (personally I think this should be built into the spec though). So why don’t we build all these great things into it if they don’t exist? We could add TODOs and calendaring and dick-sucking to email!!

Because that’s a terrible idea. People are viewing email as an application; one that has limited features and needs to be extended so it supports more than just stupid messages.

This is wrong.

We need to view email as a framework, not an application. It is used for sending messages. That’s it. It does this reliably and predictably.

Replacing email with “smarter” features will inevitably leave people out. I understand the desire to have email just be one huge TODO list. But sometimes I just want to send a fucking message, not “make a TODO.” Boom, I just “broke” the new email.

Email works because it does nothing but messaging.

How do we fix it then?

We fix it by building smart clients. Let’s take a look at some of our email-smart friends.

Outlook has built-in calendaring. BUT WAIT!!!!! Calendaring isn’t part of email!!1 No, it’s not.

Gmail has labels. You can categorize your messages by using tags essentially. Also, based on usage patterns, Gmail can give weight to certain messages. That’s not part of email either!! No, my friend, it’s not.

Xobni also has built incredible contact-management and intelligence features on top of email. How do they know it’s time to take your daily shit before you do? Defecation scheduling is NOT part of the email spec!!

How have these companies made so much fucking money off of adding features to email that are not part of email?

It’s all in the client

They do it by building smart clients! As I said, you can send any message your heart desires using email. You can send JSON messages with a TODO payload and attach a plaintext fallback. If both clients understand it, then BAM! Instant TODO list protocol. There, you just fixed email. Easy, no? Why, with the right client, you could fly a fucking space shuttle with email. That’s right, dude, a fucking space shuttle.

If your client can create a message and send it, and the receiving client can decode it, you can build any protocol you want on top of email.

That’s it. Use your imaginations. I’ll say it one more time:

There’s nothing to fix

Repeat after me: “There’s nothing to fix!” If you have a problem with email, fork a client or build your own! Nobody’s stopping you from “fixing” email. Many people have made a lot of cash by “fixing” email.

We don’t have to sit in fluorescent-lit, university buildings deliberating for hours on end about how to change the spec to fit everyone’s new needs. We don’t need 100 stupid startups “disrupting” the “broken” email system with their new protocols, that will inevitably end up being  a proprietary, non-distributed, “ad hoc, informally-specified, bug-ridden, slow implementation of half of” the current email system.

Please don’t try to fix email, you’re just going to fuck it up!! Trust me, you can’t do any better. Instead, let’s build all of our awesome new features on top of an already beautifully-working system by making smarter clients.

So I started getting this annoying error on our staging server today. I searched all over and people’s only answer was “use a 64-bit server.” Ok, but my data is less than 200MB so don’t tell me my server can’t handle it.

To make a long story short, some of the Mongo data files in the datadir were owned by root, not the “mongodb” user. I chowned them back to “mongodb” and everything went back to normal. Why this happened, I don’t know, but at least there’s a fix ;-).

1 chown -R mongodb:mongodb /srv/mongo-datadir

A while ago I created a vim highlighting script called void.vim. I’ve been using it for over a year now and just updated some things that have been bothering me recently, so feel free to check it out. This is my main color scheme I use for everything, and I created it to be be easy on the eyes but to actually look nice too. A lot of the color schemes I’ve used seem to have been really loud or have a bad choice of colors. Void is my favorite.

Here’s a sample (created with vim’s :TOhtml command):

 1 /**
 2  * Trigger an event for this object, which in turn runs all callbacks for that
 3  * event WITH all parameters passed in to this function.
 4  *
 5  * For instance, you could do:
 6  * mymodel.bind("destroy", this.removeFromView.bind(this));
 7  * mymodel.trigger("destroy", "omg", "lol", "wtf");
 8  *
 9  * this.removeFromView will be called with the arguments "omg", "lol", "wtf".
10  *
11  * Note that any trigger event will also trigger the "all" event. the idea
12  * being that you can subscribe to anything happening on an object.
13  */
14 trigger: function(ev)
15 {
16     var args   =   shallow_array_clone(Array.from(arguments));
17     [ev, 'all'].each(function(type) {
18         if(!this._events[type]) return;
19         Array.clone(this._events[type]).each(function(callback) {
20             callback.apply(this, (type == 'all') ? args : args.slice(1));
21         }, this);
22     }, this);
23 
24     return this;
25 }

This is javascript, but I also use it for HTML, CSS, php, and lisp. Note that all code highlighting on this blog is done via vim with this color scheme.

I’m currently doing some server management. My current favorite tool is TMUX, which among many other things, allows you to save your session even if you are disconnected, split your screen into panes, etc etc. If it sounds great, that’s because it is. Every sysadmin would benefit from using TMUX (or it’s cousin, GNU screen).

There’s a security flaw though. Let’s say I log in as user “andrew” and attach to my previous TMUX session: tmux attach. Now I have to run a number of commands as root. Well, prefixing every command with sudo and manually typing in all the /sbin/ paths to each executable it a pain in the ass. I know this is a bad idea, but I’ll often spawn a root shell. Let’s say I spawn a root shell in a TMUX session, then go do something else, fully intending log out later, but I forget. My computer disconnects, and I forget there’s a root shell sitting there.

If someone manages to compromise the machine, and gain access to my user account, getting a root shell is as easy as doing tmux attach. Oops.

Well, I just found out you can timeout a shell after X seconds of inactivity, which is perfect for this case. As root:

1 echo -e "\n# logout after 5 minutes of inactivity\nexport TMOUT=300\n" >> /root/.bash_profile

Now I can open root shells until my ass bleeds, and after 5 minutes of inactivity, it will log out back into my normal user account.

A good sysadmin won’t make mistakes. A great sysadmin will make mistakes self-correct ;-].

I just spent ~1 hour installing (and uninstalling) various cache plugins for WP and each of them sucking in their own special way. The main problem is that they don’t work with PHP in safe_mode, which Nearly Free Speech turns on. This is limiting because it doesn’t let PHP make directory trees on its own, the directories have to be created by hand and then have special permissions applied to them. Most caching plugins create a ton of directory trees in the wp-content/cache/ folder, rendering them useless unless for my purposes.

I just installed QuickCache which uses flat files installed into the wp-content/cache/ folder. Here was my installation:

• Upload the plugin to wp-content/plugins/
• In SSH:

  chgrp web wp-content/
  mkdir wp-content/cache
  chgrp web wp-content/cache

• Enable plugin in admin.
• Done!

Stupid easy, works really well. It seems commonplace for people to develop wordpress plugins with their error reporting set to the absolute lowest and display_errors=0. I don’t appreciate this as it usually produces broken code. I’m thankful that the QuickCache authors put in the effort to, like, actually make it work. Thanks!