UPDATE: Check out this description of PHP lambdas (much better than what I’ve done in the following).

function do_something($value)
{
    // used >= 2 times, but only in this function, so no need for a global
    $local_function = function($value) { ... };

    // use our wonderful anonymous function
    $result = $local_function($value);
    ...
    // and again
    $result = $local_function($result);
    return $result;
}

There’s also some other great stuff you can do:

$favorite_songs = array(
    array('name' => 'hit me baby one more time', 'artist' => 'britney'),
    array('name' => 'genie in a bottle', 'artist' => 'xtina'),
    array('name' => 'last resort', 'artist' => 'papa roach')
);
$song_names = array_map(function($item) { return $item['name']; }, $favorite_songs);

GnArLy bra. If PHP was 20 miles behind Lisp, it just caught up by about 30 feet. This has wonderful implications because there are a lot of functions that take a callback, and the only way to use them was to define a global function and send in an array() callback. Terrible. Inexcusable. Vomit-inducing.

Not only can you now use anonymous functions for things like array_map() and preg_replace_callback(), you can define your own functions that take functions as arguments:

function do_something_binary($fn_success, $fn_failed)
{
    $success = ...
    if($success)
    {
        return $fn_success();
    }
    return $fn_failed();
}

do_something_binary(
    function() { echo "I successfully fucked a goat!"; },
    function() { echo "The goat got away..."; }
);

Sure, you could just return $success and call whichever function you need after that, but this is just a simple example. It can be very useful to encapsulate code and send it somewhere, this is just a demonstration of the beautiful new world that just opened for PHP.

So drop your crap shared host (unless it has >= 5.3.0), get a VPS, and start using this wonderful new feature.

I did some reading and found that the preg* functions cache up to 4096 regex results in a request. This is the problem…a pretty stupid one too. It would be nice if they made this a configurable option or at least let you turn it off when, say, you are running a regex on a different string every time (why the hell would I run the same regex on the same string twice…isn’t that what variables are for?) Unless I’m misunderstanding and PHP caches the compiled regex (but not its values)…but either way, memory was climbing based on the length of the string.

Since the regex was only looking at the beginning of the string and disregarding the rest (thank god), the fix was easy (although a bit of a hack):

$val = preg_replace('/.../', '', $long_string);

Becomes:

$short_string = substr($long_string, 0, 128);
$val = preg_replace('/.../', '', $short_string);

PHP guys: how about an option to make preg* NOT have memory leaks =).

I had a few problems though. In PHP (just PHP, and nothing else) hosts were not resolving. The linux OS was resolving hosts just fine, but PHP couldn’t. It was frustrating. Also, I was unable to sudo. I kept checking permissions on all my files in /etc, rebooting, checking again, etc.

The fix

Then I looked again. /etc itself was owned by andrew:users. Huh? I changed permissions back root:root, chmod 755. Everything works. Now some background.

A while back, I wrote some software (bash + php) that makes it insanely easy to install software to several servers at once, and sync configurations for different sets of servers. It’s called “ssync.” It’s not ready for release yet, but I can say without it, I’d have about 10% of the work done that I’d finished already. Ssync is a command-line utility that lets you set up servers (host, internal ip, external ip) and create groups. Each group has a set of install scripts and configuration files that can be synced to /etc. The configuration files are PHP scriptable, so instead of, say, adding all my hosts by hand to the /etc/hosts file, I can just loop over all servers in the group and add them automatically. Same with my www group, I can add a server to the “www” group in ssync, and all of a sudden the HAproxy config knows about the server.

Here’s the problem. When ssync was sending configuration files to /etc on remote servers, it was also setting permissions on those files (and folders) by default. This was because I was using -vaz, which attempts to preserve ownership, groupship, and permissions from the source (not good). I added some new params (so now it’s “-vaz –no-p –no-g –no-o”). Completely fixed it.

You can add arrays together:

	$test1	=	array('name' => 'andrew');
	$test2	=	array('status' => 'totally gnar, dude');

	print_r($test1 + $test2);
	---------------------------
	Array
	(
	    [name] => andrew
	    [status] => totally gnar, dude
	)

Wow…who would have thought. And my most recent favorite, converting objects to events. It’s a simple foreach($object as $key => $val) and putting each element into a separate array right? WRONG:

	$array	=	(array)$object;

No fucking way. Casting actually works in this case. Why does nobody tell me anything?! This is great for parsing XML because any parser normally returns an object, and quite honestly, I hate dealing with objects. All database data is by default returned as an array usually,  and it’s a pain having some data sources being objects while others are arrays. Now it doesn’t matter…if you like objects, cast an associative array as an (object), if you like arrays cast with (array). I love PHP…

Well, PHP doesn’t have a native function for this. Why, I do not know…there seems to be functions for many other things, and adding an argument to htmlentities that returns numbered entities would seem easy enough. Either way, I wrote a quick function that takes the htmlentities translation table, adds any missing values that are not in the translation table, and runs the conversion to numbered entities. Check it:

function htmlentities_numbered($string)
{
	$table	=	get_html_translation_table(HTML_ENTITIES);
	$trans	=	array();
	foreach($table as $char => $ent)
	{
		$trans[$ent]	=	'&#'. ord($char) .';';
	}

	$trans['€']	=	'€';
	$trans['‚']	=	'‚';
	$trans['ƒ']	=	'ƒ';
	$trans['„']	=	'„';
	$trans['…']	=	'…';
	$trans['†']	=	'†';
	$trans['‡']	=	'‡';
	$trans['ˆ']	=	'ˆ';
	$trans['‰']	=	'‰';
	$trans['Š']	=	'Š';
	$trans['‹']	=	'‹';
	$trans['Œ']	=	'Œ';
	$trans['‘']	=	'‘';
	$trans['’']	=	'’';
	$trans['“']	=	'“';
	$trans['”']	=	'”';
	$trans['•']	=	'•';
	$trans['–']	=	'–';
	$trans['—']	=	'—';
	$trans['˜']	=	'˜';
	$trans['™']	=	'™';
	$trans['š']	=	'š';
	$trans['›']	=	'›';
	$trans['œ']	=	'œ';
	$trans['Ÿ']	=	'Ÿ';

	$string	=	strtr($string, $trans);
	return $string;
}

Hope it’s helpful.

UPDATE – apparently, even the numbered entities are not valid XML. Fair enough, I’ve converted them all to unicode (0×80 – 0x9F). All my ATOM feeds validate now (through feedvalidator.org).

Let me explain. PHP itself is wonderfully loosely-typed, C-like syntactically, and all around easy to write code for. The syntax is familiar because of my background. The integration with web is apparent down to its core, and it’s a hell of a lot easier than assembly to write. When perusing through a project filled to the brim with C source code, I’m usually left thinking about how it works, why the developer did what they did, and why that makes sense for that particular application. I’m usually able to figure out these questions and there’s one main reason: the code isn’t shit. With PHP, I’m usually left wondering what the developer was thinking, the 100s of ways I could have done it more efficiently, and why this person is actually making money doing this.

With roughly 90% of open-source PHP projects, everything works great. I love it, clients love it, everyone kisses eachother’s ass. But then one day you get that inevitable change request…I want it to do THIS. A quick look at the source code reveals that, omg, it’s been written by a team of highly trained ape-like creatures! It surprises me that WordPress plugins that get 100s of downloads a day throw errors (unless you disable error output, which I never do on my dev machines). Whole architectures are written with random indentation, or indentation with spaces (sorry Rubyers, but space-indentation is an evil scourge on humanity). No effort is put into separating pieces of code that could so easily be modularized if only they were given a second thought.

Do I hate PHP? No, I love PHP. I think it’s a well-written, high-level web development language. It’s fast, portable, and scalable. It allows me to focus on the problems I face, not the syntax of what I’m trying to do. Paired with an excellent editor like Eclipse (w/ PHPeclipse) I’m unstoppable. But why can’t any other PHP developers share my love of well-written code? It’s the #1 critique of PHP, and rightly so. I’m pretty sure that all programming languages, save Python, allow you to write awful, unreadable code…but PHP’s culture seems to be built around shitty code, amateurish hacks, and lack of elegance. PHP isn’t the problem, it’s the people writing it who suck!

So I do love the language, but hate most of the implementations. I have to say though, nothing is worse than Coldfusion.

So why do I hate it?

<rant>
There’s no fucking point! All bad programmers write bad code. Why create a language within a language just to force bad programmers to do one thing right? I realize that Smarty does enforce separation of logic from display very well. I’ve used it in several projects. But if its capabilities are so strikingly similar to PHP that for most things there is a 1-1 reference, why bother? Why not just use PHP code?

Also, the plugins (and {php} tag) allow you to make logical decisions, run mysql queries, send rockets to the moon…there’s nothing you can do in PHP that you cannot do in Smarty…which makes Smarty completely worthless for what it’s trying to do.

If you want to promote good programming, you don’t need Smarty. You can rewrite Smarty as a PHP object that sets variables and includes a template. I’ve written this a dozen times over…and it does the exact same thing, except templates are in PHP so everyone can understand them, there is no caching trickery going on, and best of all you don’t need to reference some stupid guide on how to display something in a strange language which you already know how to do in PHP.
</rant>

So, in summation, please don’t stop using Smarty. It’s a good piece of code for people who don’t understand the basics of separation of logic from display…but realize that Smarty is a hack, a patch, a band-aid. The REAL problem is bad programming, not something inherently wrong with PHP that needs to be rewritten.

That said, our Lyon Bros aframe framework now supports replication. It sends selects to the slave, writes to master, last_id to master, transactions to master, etc. Basically the only thing that slaves get are dumb selects (which are most of the queries anyway). I had the opportunity to test this out with the play MySQL replcation setup, and it works perfectly. It was nice to see something that complicated actually working.

Right now, it only supports connecting to two servers: master and slave (it holds off on connecting to those servers until one of them actually gets a request to save time on request startup). Basically, aframe doesn’t support load balancing. This means that if you have more than one master or slave, to use replication effectively, you’ll have to use a MySQL replication load balancer (either software or hardware). This will give you a single IP address to send requests to, but distribute the requests automatically to improve load times.

Hopefully we’ll need to set this up soon, but for now, just the one MySQL server will do =). Sometime within the next few months, we plan on releasing aframe as open source. It’s licensed and ready to go, but we have no documentation…and just saying “check out CakePHP and hope it works the same” won’t fly…especially since aframe is 23,148,855,308,184,500x better than Cake.

One thing I’d like to check out on round two of replication tinkering is Maatkit. From what I hear, it automates a lot of stuff I would be writing scripts for and checking every day. I looked at it a while ago and it seemed overly complicated, but that was even before I tried to get a server replicated. Maybe nowadays it would be easier.

That’s besides the point though. S3 chose to use SOAP, so I refuse to write my own client for it. This means that, as of late, the world is without a good free S3 uploading client. S3Fox, the firefox extension, is ok…it can’t handle SSL connections though, so expect to lose your private key to a sniffer about 10 seconds after your first request. JungleDisk is now a completely paid service (I already pay for S3, I’m not paying those guys to fucking USE it). Linux has some great command line tools for S3 (yay…), but that leaves windows with either S3fox, JD ($$$), or a handful of shitty S3 clients.

Right now, I have to use a PHP script I built around the S3 PHP Class to do any uploading that doesn’t make me vomit. The S3 class works REALLY well…it lets you assign ACL while uploading, change headers for images (for browser-side caching, mmm) and best of all, doesn’t completely suck. Let’s all thank Donovan Schönknecht for writing something that actually works well and communicates with S3.

Here’s a piece of code I wrote that wraps around the S3 uploader. It uploads images, sets Cache-Control headers, and removes the images. What you want to do is copy your images into a folder, run this file one directory up (change $start_folder to == the name of the folder your images are in), and sit back. It will upload all your images, directory structure preserved, publicly viewable and with cache-control headers.

< ?
	// quick config
	$bucket			=	'your.bucket.com';
	$start_folder	=	'images';

	// settings
	error_reporting(E_ALL);
	ini_set('display_errors', 1);
	ini_set('max_execution_time', 3600);

	// include S3 class
	include 'S3.php';
	$s3	=	new S3('[your key]', '[your secret]', false);

	// get list of files. if you don't want a subdirectory, just change this line to not need one. hopefully you know PHP...
	$files	=	recurse(array(), $start_folder);

	// loop over files and upload
	for($i = 0, $n = count($files); $i < $n; $i++)
	{
		$ext	=	preg_replace('/.*\./', '', $files[$i]);
		$type	=	'image/jpeg';
		if($ext == 'jpg')
		{
			$type	=	'image/jpeg';
		}
		else if($ext == 'gif')
		{
			$type	=	'image/gif';
		}
		else if($ext == 'png')
		{
			$type	=	'image/png';
		}

		if(
			!$s3->putObject(
				$s3->inputFile($files[$i]),
				$bucket,
				$files[$i],
				S3::ACL_PUBLIC_READ,
				array(),
				array('Cache-Control' => 'max-age=31536000', 'Content-Type' => $type)
			)
		)
		{
			echo '<span style="color:green;">Failed: upload of '. $files[$i] . '';
		}
		else
		{
			echo '<span style="color:green;">Succeeded: upload of '. $files[$i] .'';
			unlink($files[$i]);
		}
	}

	function recurse($files, $dir)
	{
		$d	=	scandir($dir);

		for($i = 0, $n = count($d); $i < $n; $i++)
		{
			if(!preg_match('/^\./', $d[$i]))
			{
				if(is_dir($dir . '/' . $d[$i]))
				{
					$files	=	recurse($files, $dir . '/' . $d[$i]);
				}
				else
				{
					$files[]	=	$dir . '/' . $d[$i];
				}
			}
		}

		return $files;
	}
?>

Feel free to modify, copy, blah blah…but give credit where it’s due. Let it be a light to you when all other lights go out. Hopefully it helps someone, because it sure helps me out.

I installed lighttpd. I already had the FastCGI setup running, not to mention I got a new Linode for testing remote PHP. The only problem was that I couldn’t load balance between my slack box (web server) and my new linode (debian app server). BTW I chose Debian because the image was smaller and from what I know, it’s essentially the same as Slack. I really haven’t had ANY problems moving to it yet, and let’s face it, Deb is a lot more standard. Installing PHP was a bitch, but that’s what apt-get is for (no, I compiled PHP…but I’ll be damned if I’m going to hand compile all the stupid dependencies).

Anyway, within 20 minutes, lighttpd had PHP running through FastCGI load balanced between two servers. Needless to say, I fell in love. Not to mention all the information I was inundated with along the way about how small and lean lighttpd is swayed this choice a little.

So as far as I know, beeets is running great on both of its “new” servers and loving it.

There was a bit of trouble getting used to the new URL rewriting scheme, but generally instead of doing apache’s

RewriteCond blahblah !-f

You can just do url-rewrite(‘(images|css|js)’ => ‘$0′)

(this is a horrible oversimplification, but you get the idea)…you write the URLs you DON’T want to be rewritten to $0. Works wonders.

All that’s left is some cache-control headers (I’m crazy about them, if you can’t tell yet), and some speed testing. I’m excited to see if lighttpd is actually faster than apache under ab.

Next up, Capistrano.

/css/css.php/file.css …which rewrites

url(/images/…) to

url(http://images.beeets.com/images/…)

And guess what, it all works. I had some bad experiences with the S3Fox firefox plugin in the past, but it’s since been updated and I’ve been using it regularly.

Also, using S3.php, all profile images now go directly onto images.beeets.com. Wicked.

So what does this mean? A few things:

1. Less bandwidth & work – beeets will spend more time serving HTML, CSS, and JS than images.

2. Safer – We were backing up profile images to S3 indirectly before, but the chances of S3 going down VS our hosting are slim.

3. Worse image caching – Before, I had .htaccess controlling all the caching for static files. I liked it that way. S3 doesn’t do this very well at all. Apparently it’s configurable, but I don’t know how…any ideas?

All in all, it should be better for beeets. Maybe we’ll actually let users have images bigger than 10×10 now ;)

Thumbs up to S3 (and probably all other Amazon web services).

Right? Yeah, right. It took two days. I can’t even really remember why. The biggest problem was that running make && make install in the mod_fastcgi source was NOT yielding a ‘mod_fastcgi.so’ as the documentation PROMISED! In fact, it installed mod_fastcgi.la instead, a highly useless file.

So how did the master get out of this bind? Beats me, try asking him. As for me, I had to run ‘ld -Bshareable *.o -o mod_fastcgi.so’ which is mentioned in some document from a long time ago in a galaxy far, far away.

Let me interject and say that the information on the FastCGI website is “not very well documented.”

Day 2. I figured, what’s the point of FastCGI if it’s not set up to connect to a remote App server? Maybe I don’t HAVE an external server set up, but we can pretend. Well that’s another nightmare. There’s a good external FastCGI guide written about it, and guess what it worked. Not really a nightmare at all, come to think of it. Quite pleasant.

All in all, shouldn’t have taken 2 days =P (I’m a tinkerer)…but fuck it, I have FastCGI now, ready to connect to all those App servers I have churning away in the background (one day).

In all the excitement, I also compiled and installed the apache worker-MPM. A few tests with ab didn’t really show any noticeable difference. But threads are cool, right?

Next up: figure out how to configure Apache to pass all requests ending in .php (whether the file exists on the web server or not) to our “app” server. Is this possible?

The client was smart enough to get FastCGI working through IIS…something I would have probably rolled over on. From then on, it was an upward battle getting a simple PHP prototype project going.

In the later versions of PHP 5, it would seem that all mssql_* functions have…been… removed? There is an ntwdblib.dll that needs to be replaced to play nicely with mssql 2005…but it doesn’t exist in the latest releases. How strange. I ended up reverting to 5.2.5, making me a not-so-bleeding-edge pushover :’(. It’s cool though.

Then MSSQL doesn’t accept normal logins, only windows ones, and it’s bloomin’ impossible finding out how to change that.

One thing Microsoft seems to have actually done right is release a rewrite module (much like mod_rewrite) that you don’t have to frickin’ pay for, which is nice. On a side note, I really hated Windows Server 2008. It’s like Vista in every way, except that the design is slightly different, somewhat. Sorry, MS, but get your shit together plz, kkthxbai.

Anyway, we got everything going. What a pain in the ass though!

If you’re wondering, I’m more of a Unix guy ;). And yes, I have used a computer before.