Saturday, December 25, 2010

Unix Sockets

There are various ways to use Unix sockets from within Erlang such as gen_socket and unixdom_drv. Code examples are even bundled with the Erlang source.

To work with Unix sockets, I've broken out the socket primitives in the procket NIF and made them accessible from Erlang.

Unix (or local or file) sockets reside as files on the local server filesystem. Like internet sockets, the Unix version can be created as either stream (reliable, connected, no packet boundary) or datagram (unreliable, packet boundaries) sockets.

Creating a Datagram Socket

The Erlang procket functions are simple wrappers around the C library. See the C library man pages for more details.

To register the server, we get a socket file descriptor and bind it to the pathname of the socket on the filesystem. The bind function takes 2 arguments, the file descriptor and a sockaddr_un. On Linux, the sockaddr_un is defined as:

typedef unsigned short int sa_family_t;

struct sockaddr_un {
    sa_family_t sun_family;         /* 2 bytes: AF_UNIX */
    char sun_path[UNIX_PATH_MAX];   /* 108 bytes: pathname */

We use a binary to compose the structure, zero'ing out the unused portion:

#define UNIX_PATH_MAX 108
#define PATH <<"/tmp/unix.sock">>

<<?PF_LOCAL:16/native,        % sun_family
  ?PATH/binary,               % address

This binary representation of the socket structure has a portability issue. For BSD systems, the first byte of the structure holds the length of the socket address. The second byte is set to the protocol family. The value for UNIX_PATH_MAX is also smaller:
typedef __uint8_t   __sa_family_t;  /* socket address family */

struct sockaddr_un {
    unsigned char   sun_len;    /* 1 byte: sockaddr len including null */
    sa_family_t sun_family;     /* 1 byte: AF_UNIX */
    char    sun_path[104];      /* path name (gag) */
The binary can be built like:
#define UNIX_PATH_MAX 104
#define PATH <<"/tmp/unix.sock">>

  (byte_size(?PATH)):8,         % socket address length
  ?PF_LOCAL:8,                  % sun_family
  ?PATH/binary,                 % address

The code below might need to be adjusted for BSD. Or it might just work. Some code I tested on Mac OS X just happened to work, presumably because the length field was ignored, the endianness happened to put the protocol family in the second byte and the extra 4 bytes was truncated.

Here is the code to send data from the client to the server:

Start up an Erlang VM and run the server (remembering to include the path to the procket library):

$ erl -pa /path/to/procket/ebin
Erlang R14B02 (erts-5.8.3) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.8.3  (abort with ^G)
1> unix_dgram:server().

And in a second Erlang VM run:
1> unix_dgram:client(<<104,101,108,108,111,32,119,111,114,108,100>>). % Erlangish for <<"hello world">>, I am being a smartass

In the first VM, you should see printed out:
<<"hello world">>

Creating an Abstract Socket

Linux allows you to bind an arbitrary name (a name that is not a file system path) by using an abstract socket. The abstract socket naming convention uses a NULL prefacing arbitrary bytes in place of the path used by traditional Unix sockets. To define an abstract socket, a binary is passed as the second argument to procket:bind/2, in the format of a struct sockaddr:
<<?PF_LOCAL:16/native,        % sun_family
  0:8,                        % abstract address
  "1234",                     % the address

To create a datagram echo server, the source address of the client socket is bound to an address so the server has somewhere to send the response. We modify the datagram server to use recvfrom/4, passing in an additional flag argument (which is set to 0) and a length. recvfrom/4 will return an additional value containing up to length bytes of the socket address.

We also need to modify the client to bind to an abstract socket. The server will receive this socket address in the return value of recvfrom/4; this value can be passed to sendto/4.

1> unix_dgram1:server().

1> unix_dgram1:client(<<104,101,108,108,111,32,119,111,114,108,100>>).
<<"hello world">>

Creating a Stream Socket

To create a stream socket, we use the SOCK_STREAM type (or 1) for the second value passed to socket/3. The socket arguments can be either integers or atoms; for variety, atoms are used here.

After the socket is bound, we mark the socket as listening and poll it (rather inefficiently) for connections. When a new connection is received, it is accepted, the file descriptor for the new connection is returned and a process is spawned to handle the connection.

On the client side, after obtaining a stream socket, we do connect the socket and so do not need to explicitly bind it.

Running the same steps for the client and server as above:

1> unix_stream:server().
<<"hello world">>
** client disconnected

1> unix_stream:client(<<104,101,108,108,111,32,119,111,114,108,100>>).
<<"hello world">>

Friday, December 3, 2010

ICMP Ping in Erlang, part 2

I've covered sending ICMP packets from Erlang using BSD raw sockets and Linux's PF_PACKET socket option.

gen_icmp tries to be a simple interface for ICMP sockets using the BSD raw socket interface for portability. It should work on both Linux and BSD's (I've tested on Ubuntu and Mac OS X).

Sending Ping's

To ping a host:
1> gen_icmp:ping("").
      <<" !\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJK">>}}]
The response is a list of 3-tuples. The third element is a 2-tuple holding the ICMP echo request ID, the sequence number, the elapsed time and the payload.

A bad response looks like:
2> gen_icmp:ping("").
The argument to gen_icmp:ping/1 takes either a string or a list of strings. For example, to ping every host on a /24 network:
1> gen_icmp:ping([ {192,168,213,N} || N <- lists:seq(1,254) ]).
gen_icmp:ping/1 takes care of opening and closing the raw socket. This operation is somewhat expensive because Erlang is spawning a setuid executable to get the socket. If you'll be doing a lot of ping's, it's better to keep the socket around and use ping/3:
1> {ok,Socket} = gen_icmp:open().

2> gen_icmp:ping(Socket, ["", "", {192,168,213,1}],
    [{id, 123}, {sequence, 0}, {timeout, 5000}]).
      <<" !\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJK">>}},
      <<" !\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJK">>}},
      <<" !\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJK">>}}]

3> gen_icmp:close(Socket).

Creating Other ICMP Packet Types

ICMP destination unreachable and time exceeded packets return at least the first 64 bits of the header and payload of the original packet. Here is an example of generating an ICMP port unreachable for a fake TCP packet sent to port 80.

-define(IPV4HDRLEN, 20).
-define(TCPHDRLEN, 20).

unreachable(Saddr, Daddr, Data) ->
    {ok, Socket} = gen_icmp:open(),

    IP = #ipv4{
        p = ?IPPROTO_TCP,
        len = ?IPV4HDRLEN + ?TCPHDRLEN + byte_size(Data),
        saddr = Daddr,
        daddr = Saddr

    TCP = #tcp{
        dport = 80,
        sport = crypto:rand_uniform(0, 16#FFFF),
        seqno = crypto:rand_uniform(0, 16#FFFF),
        syn = 1

    IPsum = epcap_net:makesum(IP),
    TCPsum = epcap_net:makesum([IP, TCP, Data]),

    Packet = <<
        (epcap_net:ipv4(IP#ipv4{sum = IPsum}))/bits,
        (epcap_net:tcp(TCP#tcp{sum = TCPsum}))/bits,

    ICMP = gen_icmp:packet([
        {type, ?ICMP_DEST_UNREACH},
        {code, ?ICMP_UNREACH_PORT}
    ], Packet),

    ok = gen_icmp:send(Socket, Daddr, ICMP),

To create the IPv4 and TCP headers, we make the protocol records and use the epcap_net module functions to encode the headers with the proper checksums. For creating the ICMP packet, we use the gen_icmp:packet/2 function (which again simply calls epcap_net).

ICMP Ping Tunnel

We can tunnel any data we like in the payload of an ICMP packet. In this example, we'll use ICMP echo requests to tunnel an ssh connection between 2 hosts. The ICMP echo replies sent back by the peer OS ensure the data was received, like the ACK in a TCP connection.

The tunnel exports 2 functions:

  • ptun:server(ClientAddress, LocalPort) -> void()

    Types   ClientAddress = tuple()
                LocalPort = 0..65534
        ClientAddress is the IPv4 address of the peer represented as a tuple.
        The server listens on LocalPort for TCP connections and will close
        the port after a TCP client connects.  Data received on this port
        will be sent to the peer as the payload of the ICMP packets.

  • ptun:client(ServerAddress, LocalPort) -> void()

    Types   ServerAddress = tuple()
                LocalPort = 0..65534
        ServerAddress is the IPv4 address of the peer.
        When the client receives an ICMP echo request, the client opens a
        TCP connection to the LocalPort on localhost, proxying the data.
To start the tunnel, you'll need 2 hosts. In this example, is the client and is the server. forwards any tunnelled data it receives to a local SSH server. On
1> ptun:client({192,168,213,119}, 22).
1> ptun:server({192,168,213,7}, 8787).
Open another shell and start an SSH connection to port 8787 on localhost:
ssh -p 8787
$ ifconfig eth0 | awk '/inet addr/{print $2}'