System Call Debugging with (s|k|d(trace|truss)) (that's strace, ktrace, dtrace or dtruss)

Recently while working on getting tests done for Catalyst::Devel so it could be shipped, and I absolutely could not figure out how to get the test for the development server to work. So, I queried #catalyst-dev about it, and my good friend mst told me to use strace to figure out what was going on behind the scenes. In this case, I used dtruss since I'm a Mac user and for some reason the usual *nix tool doesn't work on the Mac platform.

Basically, what went down was this:

1. sudo dtruss -f prove -l t/author/optional_http-server.t
2. a whole load of rather cryptic output (see below for a snippet)
3. analysis and correction

What I found was that since the Catalyst development server is chock full of system calls that are chock full of information on what's going on (and going wrong) behind the scenes, dtruss was the perfect tool for grabbing messages being relayed between processes and not necessarily being spewed out to my terminal. Here's a small snippet of the problem area I (well, mst actually) discovered:

46665/0x117e04: stat64("/opt/local/lib/perl5/vendorperl/Catalyst/ScriptRunner.pm\0", 0x7FFF5FBFEE90, 0xFFFFFFFFFFFFFFFF) = -1 Err#2 46665/0x117e04: stat64("/opt/local/lib/perl5/5.8.9/darwin-2level/Catalyst/ScriptRunner.pmc\0", 0x7FFF5FBFEF20, 0x0) = -1 Err#2 46665/0x117e04: stat64("/opt/local/lib/perl5/5.8.9/darwin-2level/Catalyst/ScriptRunner.pm\0", 0x7FFF5FBFEE90, 0xFFFFFFFFFFFFFFFF) = -1 Err#2 46665/0x117e04: stat64("/opt/local/lib/perl5/5.8.9/Catalyst/ScriptRunner.pmc\0", 0x7FFF5FBFEF20, 0x0) = -1 Err#2 46665/0x117e04: stat64("/opt/local/lib/perl5/5.8.9/Catalyst/ScriptRunner.pm\0", 0x7FFF5FBFEE90, 0xFFFFFFFFFFFFFFFF) = -1 Err#2 46665/0x117e04: stat64("./Catalyst/ScriptRunner.pmc\0", 0x7FFF5FBFEF20, 0x0) = -1 Err#2 46665/0x117e04: stat64("./Catalyst/ScriptRunner.pm\0", 0x7FFF5FBFEE90, 0xFFFFFFFFFFFFFFFF) = -1 Err#2 46665/0x117e04: writenocancel(0x2, "Can't locate Catalyst/ScriptRunner.pm in @INC (@INC contains: /Users/dhoss/web-devel/Cat-Runtime-5.8/t/author/../../lib /Users/dhoss/web-devel/Cat-Runtime-5.8/lib /opt/local/lib/perl5/siteperl/5.8.9/darwin-2level /opt/local/lib/perl5/siteperl/5.8.9 /opt/", 0x2D0)

That's just a snippet. Somewhat cryptic, but it's all right there. Check out this line: 46665/0x117e04: writenocancel(0x2, "Can't locate Catalyst/ScriptRunner.pm in @INC (@INC contains: /Users/dhoss/web-devel/Cat-Runtime-5.8/t/author/../../lib /Users/dhoss/web-devel/Cat-Runtime-5.8/lib /opt/local/lib/perl5/siteperl/5.8.9/darwin-2level /opt/local/lib/perl5/site_perl/5.8.9 /opt/", 0x2D0). Derp! I was running this test in Catalyst::Runtime trunk/ that doesn't have my ScriptRunner classes merged yet. So, I reran this test (using dtruss -f again) under my branch, found out that the options being passed to the development server were incorrect, corrected, svn committed, and can now take another step (finally) toward shipping Catalyst::Devel!