The Composer.js MVC framework has just released version 1.0! Note that this is a near drop-in replacement for Composer v0.1.x.

There are some exciting changes in this release:

  • Composer no longer requires Mootools… jQuery can be used as a DOM backend instead. In fact, it really only needs the selector libraries from Moo/jQuery (Slick/Sizzle) and can use those directly. This means you can now use Composer in jQuery applications.
  • Controllers now have awareness of more common patterns than before. For instance, controllers can now keep track of sub-controllers as well as automatically manage bindings to other objects. This frees you up to focus on building your app instead of hand-writing boilerplate cleanup code (or worse, having rogue objects and events making your app buggy).
  • The ever-popular RelationalModel and FilterCollection are now included by default, fully documented, and considered stable.
  • New class structures in the framework expose useful objects, such as Composer.Class which gives you a class structure to build on, or Composer.Event which can be used as a standalone event bus in your app.
  • There’s now a full test suite so people who want to hack away on Composer (including us Lyon Bros) can do so without worrying about breaking things.
  • We updated the doc site to be much better organized!

Breaking changes

Try as we might, we couldn’t let some things stay the same and keep a clear conscience. Mainly, the problems we found were in the Router object. It no longer handles hashbang (#!) fallback…it relies completely on History.js to handle this instead. It also fixes a handful of places where non-idiomatic code was used (see below).

  • Composer.Router: the on_failure option has been removed. Instead of
    var router = new Composer.Router(routes, {on_failure: fail_fn});

    you do

    var router = new Composer.Router(routes);
    router.bind('fail', fail_fn);

  • Composer.Router: The register_callback function has been removed. In order to achieve the same functionality, use router.bind('route', myfunction);.
  • Composer.Router: The “preroute” event now passes {path: path} as its argument instead of path. This allows for easier URL rewriting, but may break some apps depending on the old method.
  • Composer.Router: History.js is now a hard requirement.

Sorry for any inconvenience this causes. However, since the rest of the framework is backwards compatible, you should be able to just use the old Composer.Router object with the new framework without any problems if you don’t wish to convert your app.

Have fun!

Check out the new Composer.js, and please open an issue if you run into any problems. Thanks!

- The Lyon Bros.

Comments Off

I recently embarked on a project to rebuild the main functionality of Turtl in common lisp. This requires embedding lisp (using ECL) into node-webkit (or soon, Firefox, as node-webkit is probably getting dumped).

To allow lisp and javascript to communicate, I made a simple messaging layer in C that both sides could easily hook into. While this worked, I stumbled on nanomsg and figured it couldn’t hurt to give it a shot.

So I wrote up some quick bindings for nanomsg in lisp and wired everything up. So far, it works really well. I can’t tell if it’s faster than my previous messaging layer, but one really nice thing about it is that it uses file descriptors which can be easily monitored by an event loop (such as cl-async), making polling and strange thread < –> thread event loop locking schemes a thing of the past (although cl-async handles all this fairly well).

This simplified a lot of the Turtl code, and although right now it’s only using the nanomsg “pair” layout type, it could be easily expanded in the future to allows different pieces of the app to communicate. In other words, it’s a lot more future-proof than the old messaging system and probably a lot more resilient (dedicated messaging library authored by 0MQ mastermind beats hand-rolled, hard-coded simple messaging built by non-C expert).

Lately I’ve been neck-deep in embedding. Currently, I’m building a portable (hopefully) version of Turtl‘s core features in ECL.

Problem is, when embedding turtl-core into Node-webkit or Firefox, any output that ECL writes to STDOUT triggers:

C operation (write) signaled an error. C library explanation: Bad file descriptor.

Well it turns out Windows doesn’t let you write to STDOUT unless a console is available, and even if using msys, it doesn’t create a console for GUI apps. So here’s a tool (in lisp, of course) that will let you convert an executable between GUI and console.

Seems to work great. Special thanks to death.

Comments Off

It can be nice to access your FF extension’s variables/functions from the browser console (ctrl+shift+j) if you need some insight into its state.

It took me a while to figure this out, so I’m sharing it. Somewhere in your extension, do:

var chromewin = win_util.getMostRecentBrowserWindow();
chromewin.my_extension_state = ...;

Now in the browser console, you can access whatever variables you set in the global variable my_extension_state. In my case, I used it to assign a function that lets me evaluate code in the addon’s background page. This lets me gain insight into the background page’s variables and state straight from the browser console.

Note! This is a security hole. Only enable this when debugging your extension/addon. Disable it when you release it.

Note that this may or may not work on your device. If you’re running into an app that works in a real browser but on in your Android’s stock browser, do this:

  1. Navigate to your app in the browser.
  2. In the same tab go to about:debug
  3. Reload (it may reload for you).
  4. Profit.

This will show you errors that even window.onerror doesn’t catch, which should help you narrow down your problem(s).

Source: This stackoverflow answer.

Comments Off

As you all know, I’m building a Turtl, a browser extension for client-side encrypted note/file storage.

Well once in a while the I need to debug the release version. There are docs scattered around detailing how to do this, but as usual with this type of thing you really need to do some digging.

By default, Firefox’s Browser Console only logs error events. You can change this to log any and all console.log() calls from your addon (and any other addon) by doing this:

  1. Go to about:config
  2. Search for the key extensions.sdk.console.logLevel
  3. If it exists, set it to “info”, otherwise add a new string with the key extensions.sdk.console.logLevel and the value “info”

Boom, all your addon’s log calls now show up in the browser console.

Comments Off

I use curl to test out my HTTP libraries all the time. Recently, I ran into an issue where when uploading a file (25mb) from curl in the command line to my common lisp app server, only about half the data showed up (12.5mb). I was doing this:

curl -H 'Authorization: ...' -H 'Transfer-Encoding: chunked' --data-binary @/media/large_vid.mov

Naturally, I assumed the issue was with my libraries. It could be the cl-async library dropping packets, it could be the HTTP parser having issues, and it could be the app server itself. I mean, it has to be one of those. Curl has been around for ages, and there’s no way it would just drop data. So I spent days tearing my hair out.

Finally, I ran curl with the --trace and looked at the data. It provides a hex dump of everything it sends. It’s not formatted perfectly, but with vim’s block select and a few handy macros, I was able to get the length of the data being sent: 12.5mb. That’s right, curl was defying me. There was no error in my code at all.

I did a search online for curl not sending the full file data when using --data-binary. Nothing. So I looked over my options and found -T which looks surprisingly similar to --data-binary with the @ modifier. I tried:

curl -H 'Authorization: ...' -H 'Transfer-Encoding: chunked' -T /media/large_vid.mov

All 25mb came through (every byte accounted for).

Conclusion

If you’re uploading files, use -T /path/to/file instead of --data-binary @/path/to/file. Note that -d/-D were also “broken.”

Comments Off

Hi FORKS. Tuesday I announced my new app, Turtl for Chrome (and soon Firefox). Turtl is a private Evernote alternative. It uses AES-256bit encryption to obscure your notes/bookmarks before leaving the browser. What this means is that even if your data is intercepted on the way to the server or if the server itself is compromised, your data remains private.

Even with all of Turtl’s privacy, it’s still easy to share boards with friends and colleagues: idea boards, todo lists, youtube playlists, etc. With Turtl, only you and the people you share with can see your data. Not even the guys running the servers can see it…it’s just gibberish without the key that you hold.

One more thing: Turtl’s server and clients are open-source under the GPLv3 license, meaning anyone can review the code or use it for themselves. This means that Turtl can never be secretly compromised by the prying hands of hackers or government gag orders. The world sees everything we do.

So check out Turtl and start keeping track of your life’s data. If you want to keep up to date, follow Turtl on Twitter.

I recently installed FreeBSD 9.1-RELEASE on a VirtualBox VM to do some cl-async testing. I wanted to get Xorg running so I could edit code at a more “comfortable” resolution. I was able to get Xorg running fairly easily just by installing Xfce from /usr/ports.

However, upon starting Xorg, my keyboard mouse would not work. I tried many things: following the steps in the handbook, enabling/disabling hald, reconfiguring Xorg, etc. No luck. My Xorg.0.log was telling me that it couldn’t load the kdb/mouse drivers. After snooping around some forums, I found the solution:

  • Install the x11-drivers/xf86-input-keyboard port
  • Install the x11-drivers/xf86-input-mouse port

After doing this, all was right with the world. Just to clarify, I am using dbus/hald and more or less using the default configuration that Xorg -configure gave me.

Comments Off

A while ago, I released cl-async, a library for non-blocking programming in Common Lisp. I’ve been updating it a lot over the past month or two to add features and fix bugs, and it’s really coming along.

My goal for this project is to create a general-purpose library for asynchronous programming in lisp. I think I have achieved this. With the finishing of the futures implementation, not only is the library stable, but there is now a platform to build drivers on top of. This will be my next focal point over the next few months.

There are a few reasons I decided to build something new. Here’s an overview of the non-blocking libraries I could find:

  • IOLib – An IO library for lisp that has a built-in event loop, only works on *nix.
  • Hinge – General purpose, non-blocking library. Only works on *nix, requires libev and ZeroMQ.
  • Conserv – A nice layer on top of IOLib (so once again, only runs on *nix). Includes TCP client/server and HTTP server implementations. Very nice.
  • teepeedee2 – A non-blocking, performant HTTP server written on top of IOLib.

I created cl-async because of all the available libraries, they are either non-portable, not general enough, have too many dependencies, or a combination of all three. I wanted a library that worked on Linux and Windows. I wanted a portable base to start from, and I also wanted tools to help make drivers.

Keeping all this in mind, I created bindings for libevent2 and built cl-async on top of them. There were many good reasons for choosing libevent2 over other libraries, such as libev and libuv (the backend for Node.js). Libuv would have been my first choice because it supports IOCP in Windows (libevent does not), however wrapping it in CFFI was like getting a screaming toddler to see the logic behind your decision to put them to bed. It could have maybe happened if I’d written a compatibility layer in C, but I wanted to have a maximum of 1 (one) dependency. Libevent2 won. It’s fast, portable, easy to wrap in CFFI, and on top of that, has a lot of really nice features like an HTTP client/server, TCP buffering, DNS, etc etc etc. The list goes on. That means less programming for me.

Like I mentioned, my next goal is to build drivers. I’ve already built a few, but I don’t consider them stable enough to release yet. Drivers are the elephant in the room. Anybody can implement non-blocking IO for lisp, but the real challenge is converting everything that talks over TCP/HTTP to be async. If lisp supported coroutines, this would be trivial, but alas, we’re stuck with futures and the lovely syntax they afford.

I’m going to start with drivers I use every day: beanstalk, redis, cl-mongo, drakma, zs3, and cl-smtp. These are the packages we use at work in our queue processing system (now threaded, soon to be evented + threaded). Once a few of these are done, I’ll update the cl-async drivers page with best practices for building drivers (aka wrapping async into futures). Then I will take over the world.

Another goal I have is to build a real HTTP server on top of the bare http-server implementation provided by cl-async. This will include nice syntax around routing (allowing REST interfaces), static file serving, etc.

Cl-async is still a work in progress, but it’s starting to become stabilized (both in lack of bugs and the API itself), so check out the docs or the github project and give it a shot. All you need is a lisp and libevent =].