Skip to content

Categories:

Some performance monitoring stuff that’s been on my radar lately

Monitoring server load on Linux

I’m all about top and mytop, but there’s at least one surprise in here for me — namely, /usr/bin/tload.

Measuring & Optimizing I/O Performance using iostat…

If IO performance is suspect, iostat is your best friend. Having said that, the man pages are cryptic so don’t be surprised if you find yourself reading the source. To get started, identify the device in question and start a monitoring process…

which contains a note on how Facebook stores 360 photo uploads every second.

Posted in Server.

astalavista.com compromised

Anybody who’s had an interest in security over the last 10 years has probably heard of astalavista.net — kind of a clearinghouse for all sorts of security-related info, vulnerability announcements, etc.

Here’s a blow-by-blow transcript of their systems getting completely wiped out by someone who didn’t agree with their practices. The transcript is hosted on pastebin.com but was originally available as the index.html of the site itself.

It looks like they used custom code to exploit a hole in astalavista’s Litespeed webserver; googling the exploit used just brings up copies of the transcript itself.

This is probably the most brutal exploit transcript I’ve ever seen — reference line 1839 where the exploiter, having found where the backups are being stored by trawling the admin’s .bash_history, FTPs there and deletes their offsite backups. (If your production system stores the FTP password or password-less SSH keys to your offsite backup server, exposed through your .bash_history no less, is it really offsite?)

astalavista is still down at the moment; I imagine they’ve scrambled to change all their passwords, shore up their exposure, and see if they can dig up a backup from anywhere.

It makes a pretty remarkable contrast with this recent How I Hacked Hacker News (with arc security advisory) accounting by someone who compromised Paul Graham’s news.ycombinator.com via some pretty painstaking research of system behavior — and then worked with Robert T. Morris (the Robert T. Morris) to get the issue fixed.

Posted in Server. Tagged with .

HOWTO: Rack Middleware for API Throttling

I will show you a technique to impose a rate limit (aka API Throttling) on a Ruby Web Service. I will be using Rack middleware so you can use this no matter what Ruby Web Framework you are using, as long as it is Rack-compliant.

Awesome. I can think of so many uses for this basic pattern to replace stuff I usually have to do at the app level (slow) or at the Apache level (difficult to port, prone to decay).

Comments note that Memcache might be a better pick than Redis, which works even better with the setup I have in mind, though Redis sounded fun too.

Posted in Server. Tagged with , .

HOWTO: Totally Unmathematical Chili

Here’s the basic skeleton of the recipe I use for chili. I started a few years ago with something called “Touchdown Chili” that a co-worker printed out of an online magazine, and over the years I’ve tweaked it relentlessly. I still have the original print-out that I pull out and use as a rough guide, even though my version has changed so much the printout is basically a series of phase cues — i.e., “time for spices”, “ok, do the meat”, “now you should be about finished adding things”, etc.

Part of the joy of this aboriginal approach is that it’s different every time.

A note on the “Do beans belong in chili?” holy war: Personally, I have no particular use for pulses, though sometimes I’ll make a sidecar batch for my fiancee (who is vegetarian) that includes a robust mix of black beans, white beans and kidney beans. She likes it. I make no specific recommendation in this regard and consider this momentous personal decision best resolved via quiet introspection and discreet consultation of sympathetic clergy members or other local community leaders.

Continued…

Posted in Random. Tagged with .

Making emacs’ word navigation respect underscores everywhere

I’ve been playing with emacs lately, using technomancy’s emacs starter kit. It’s awesome so far, but some piece of the (rather large) config fileset is clobbering what I’d consider normal navigation behavior.

I work with a ton of stuff that uses underscores, especially SQL, and it’s been driving me crazy that, on its own, my emacs config doesn’t consider the underscore punctuation in all modes, so hitting M-DEL (Alt+Backspace) to delete back one word in sqlplus mode deletes the whole string — underscores and all.

That is, if | is our cursor, and we’re inside emacs at a sqlplus prompt like so…

desc prod.some_long_table_name|

hitting Alt+Backspace once gets you…

desc prod.|

which is annoying, especially because emacs’ autocomplete is so awesomely useful in sqlplus mode. I’d guess most database schemas have lots of tablenames that share prefixes (e.g., user_history, user_bookmarks, user_buddies, etc), which renders that autocomplete somewhat less than effective.

I’d much rather have it so that the cursor stopped at underscores. On successive keypresses, you’d zap back another segment, stopping at the underscore, like so…

desc prod.some_long_table_name|
desc prod.some_long_table_|
desc prod.some_long_|
desc prod.some_|
desc prod.|

This is already the long-established behavior of interfaces that inherit from emacs, like BASH’s emacs mode, and irssi, but is evidently a legacy design decision within Emacs without an obvious config handle, nearly undiscussed as far as I could google…

From a comment in python-mode.el:

;; For historical reasons, underscore is word class instead of
;; symbol class. GNU conventions say it should be symbol class, but
;; there’s a natural conflict between what major mode authors want
;; and what users expect from `forward-word’ and `backward-word’.
;; Guido and I have hashed this out and have decided to keep
;; underscore in word class. If you’re tempted to change it, try
;; binding M-f and M-b to py-forward-into-nomenclature and
;; py-backward-into-nomenclature instead. This doesn’t help in all
;; situations where you’d want the different behavior
;; (e.g. backward-kill-word).

Luckily, TFM is rather nice.

The short answer is: one needs to specify the underscore character as a member of emacs’ punctuation class…

(modify-syntax-entry ?_ "_")

That will only work for text-mode, and gets overwritten on a major-mode switch by any new syntax table specified, so the following hook reference is necessary for this change to be global for all major modes…

(defun change-major-mode-hook ()
(modify-syntax-entry ?_ "_"))

Folks who aren’t emacs newbs might be aware of further subtleties, but this seems to produce exactly what I want — autocomplete and word navigation in every mode that’s identical to the other command-line interfaces I use.

Posted in Development. Tagged with .

TODO

HOWTO: Autocomplete fieldtypes in Expression Engine.
I recently had need to provide an autocomplete fieldtype in EE’s weblog UI. Nifty, and jQuery made it very simple. I need to package this up according to EE convention, document it, and release it to the EE community.

Rails to Expression Engine and back, from a data modelling perspective.
While there’s definitely something to be said for handing a customer a solution built on something as tried-and-true as mod_php, EE has made me realize how spoiled frameworks in general (and Rails in particular) makes us (me) when expressing data relationships. Hopefully EE 2.0 comes out and obsoletes this idea before I type it up, though I expect it to be not just fun but also educational for the whole family.

Posted in Development. Tagged with , .

My RSS Feeds

An ungainly dump of what’s currently in my RSS feeds. Admin/utility/monitoring feeds snipped for your sanity.

Helpful usage hints: feeds best enjoyed via rbot’s RSS module when home, via Google Reader from BlackBerry when not.

Friends With Cool Blogs

Chris Patterson’s Skull Jackpot
- You might eyeball this and conclude Chris was only a gangster in the UI arena, but you’d be wrong.

Ben Menoza
- Ben is a user experience jet-setter who has some very interesting project-lifecycle notes I hope he finishes documenting.

Dev, Ruby/Rails…

Tom Moertel’s Weblog
- Quality rants on programming theory and stuff geeks like

PlanetRails
- Yet another Planet- community aggregator.

MotionStandingStill
- Performance Rails stuff.

railscasts
- Free railscasts all the time. Awful nice of them.

Continued…

Posted in Random. Tagged with , , .

Rules of Database App Aging

Rules of Database App Aging

All Fields Become Optional

As your dataset grows, exceptions creep in. There’s not enough research time to fill in all your company profiles, there’s one guy in Guam when you expected everyone to be in a U.S. state, there’s data missing from the page you’re scraping, you have to pull updates from a new source.

Every field eventually loses that beautiful NOT NULL sheen, your code gets filled up with guard clauses of one kind or another and every

in your template is wrapped by an if statement. And this happens to foreign keys, too, so OR IS (NOT) NULL sneaks in and left joins mutate into outer joins.

Posted in Server. Tagged with .

Dynamic, Search-based RSS feeds, and thinking_sphinx + HAML

Dynamic, Search-based RSS feeds

“Users have the ability to search for a lot of different information, and I needed to provide them with a way to save that search via an RSS feed. To see how I went about performing basic search, see here.

In my controller, I started by adding the ability to respond_to rss…”

Posted in Development, Server. Tagged with .

A few extremely cool things…

Mibbit is an awesome example of a guy who found a niche, really thought about what it was missing, and filled it expertly. Every time I see this app, I’m secretly jealous I didn’t write it.

Github — social media meets coding. Instantly makes rubyforge look like tucows.com circa 1996. A good example of filling a need you didn’t know existed. Getting insight into others’ development projects via ‘following’ developers and projects — or via RSS — turns out to be oddly addictive, and turns up neat little things like this and this, which couldn’t have really existed before the github model except as a fractured melodramatic irc log somewhere. I don’t have a pressing need for any private repos anytime soon but am compelled to upgrade my account out of sheer gratitude.

Lastly, I’ve finally checked out jQuery, and can now say: I get why all the fuss.

Posted in Random. Tagged with .