When the Bugs Have Bugs

A few months ago, I found this. Compiling a regular expression would crash beam.

N = 819,
re:compile([lists:duplicate(N, $(), lists:duplicate(N, $))]).

After going through a bit of effort, I figured out how to compile a debug version of beam. And then, of course, I discovered the clever minds behind Erlang have already thought about this and made it easy. Essentially, after compiling Erlang:

# Recommended if you are a vi user
# Yes, the debugger forces you to use emacs
cat >> ~/.emacs
(setq viper-mode t)
(require 'viper)
^D

export ERL_TOP=$(pwd)
cd erts/emulator
make debug FLAVOR=plain # or smp
cd ~-
bin/cerl -debug -gdb # -smp

After reading through the source code and adding a few printf's, I tracked the bug down to an incorrect test in PCRE. The magic number (819) apparently comes from:

819 x 5 bytes (capturing bracket) + 3 bytes (opening bracket) = 4098 bytes

The compile workspace is 4096 bytes, so there is a 2 byte overflow. Well, today Phillip Hazel, the author of PCRE, corrected the bug. Awesome!! Thanks, Phillip!
So here I am making the world safer one bug at a time, preparing a patch for Erlang. Except when I went to test the fix on Mac OS X, beam crashed. Ouch. This time:

% works!
N = 611,
re:compile([lists:duplicate(N, $(), lists:duplicate(N, $))]).

% booo! crashes!!
N = 612,
re:compile([lists:duplicate(N, $(), lists:duplicate(N, $))]).

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0xb014effc
[Switching to process 3601]
0x001c04e4 in compile_branch (optionsptr=0x0, codeptr=0x0, ptrptr=0x0, errorcodeptr=0x0, firstbyteptr=0x0, reqbyteptr=0x0, bcptr=0x0, cd=0x0\
        , lengthptr=0x0) at pcre_compile.c:2355

Except, beam didn't crash when running inside gdb. I figured out the debug beam was non-smp and, after compiling a debug smp version, I got the longest backtrace EVAH.

Yet the same code works with an SMP Erlang on Solaris.
Blah, debugging threaded code is a pain. If someday, someone figures out how to do something malicious with this, please send me a postcard from whatever island retreat you've purchased with all your stolen credit cards or DoS extortions.