Using ld to manually build an executable

Author's note: further work on raop_play has been sporadic due to other happenings, though I have some new results addressing song truncation, playing other file types, and playback of mono files. I will (hopefully) get this posted shortly!

About three days ago, I finally bit the bullet and decided to give Kubuntu a try. While I have been using Debian happily for the past few years, I am discovering an increasing desire for simplicity and to have things work out of the box, rather than spend large amounts of time configuring everything from scratch. One of my side-goals was to try out the Pulse audio system to see if I could successfully integrate my Linux sound with my Airport Express. As it turns out, this does not work as smoothly as I'd anticipated. While getting it up and running was mostly straightforward, there are some playback issues that appear to be unresolved within the community. Perhaps a project for another day, but with a freshly-installed OS I wanted to play some tunes NOW!

So instead I go back to my previous work on raop_play and simply recompile with the new libraries on this installation... but I run into a problem. For some reason, the OpenSSL library wasn't being found by the linker, giving me the following error:

...

gcc -o raop_play raop_play.o raop_client.o rtsp_client.o aexcl_lib.o base64.o aes.o m4a_stream.o audio_stream.o wav_stream.o mp3_stream.o flac_stream.o ogg_stream.o aac_stream.o pls_stream.o pcm_stream.o -lssl -lsamplerate -lid3tag
raop_client.o: In function `rsa_encrypt':
raop_client.c:(.text+0x131): undefined reference to `RSA_new'
raop_client.c:(.text+0x175): undefined reference to `BN_bin2bn'
raop_client.c:(.text+0x1b3): undefined reference to `BN_bin2bn'
...

... and so on for another dozen lines. Actually, it was an even-worse problem at first, until I discovered that gcc is sensitive to the ordering of files and library references, and fixed the Makefile to move all "-l" options to the end of the command, as shown above.

I racked my brain, wondering if I had an incompatible version of the OpenSSL library, or if the system's library path was broken, but every workaround I tried yielded the same result.


As a baseline, I decided to try a simple test program:

#include <openssl/rsa.h>
int main(int argc, char **argv) {
  RSA *r;
  r=RSA_new();
  return 0;
}

Again, I got an error during compilation:

$ gcc test.c -lssl
/tmp/cc8VZDWR.o: In function `main':
test.c:(.text+0xa): undefined reference to `RSA_new'
collect2: ld returned 1 exit status

But then, on a whim, I attempted to break apart compilation and linking:

$ gcc -c test.c

$ ld test.o -lssl
ld: warning: cannot find entry symbol _start; defaulting to 00000000080481d0

No warning due to missing libraries! I ought to be able to use this brute-force method to compile and link raop_play as well. However, this ld warning was a problem. Clearly ld requires additional arguments to produce a full-fledged executable. Usually a quick Google search would turn up some guidance on how to use this; unfortunately it seems in this case most folks shy away from invoking ld by hand. An anonymous quote from one page:

"You need some more object files and libraries [to use ld directly], but I don't know which ones or where to find them. gcc knows, so use gcc."

Fortunately, I did find this page which showed an example. Apparently part of the trick is some standard code that gets linked at the beginning and ending of every program to handle initialization and termination. So for test.c, we end up with:

ld -dynamic-linker /lib/ld-linux.so.2 /usr/lib/i386-linux-gnu/crt1.o /usr/lib/i386-linux-gnu/crti.o -lc test.o -lssl /usr/lib/i386-linux-gnu/crtn.o

I've underlined the original bits to make it obvious what is added. I haven't explored what each of these pieces does (yet), but for now it suffices to say that I was able to successfully get raop_play compiled and linked using the following commands:

$ gcc -c *.c

$ ld -o raop_play -dynamic-linker /lib/ld-linux.so.2 /usr/lib/i386-linux-gnu/crt1.o /usr/lib/i386-linux-gnu/crti.o -lc *.o -lssl -lsamplerate -lid3tag /usr/lib/i386-linux-gnu/crtn.o

Here I've underlined the application-specific pieces: output file, input files, and libraries. It appears that this is a functional template for linking any object files, as long as exactly one of them was compiled with a "main" function in it.

Update 8/1/2014: I recently needed to build this application on a 64-bit system, and discovered that several of the standard linked files have different names or paths (shown in bold):

$ ld -o raop_play -dynamic-linker /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 /usr/lib/x86_64-linux-gnu/crt1.o /usr/lib/x86_64-linux-gnu/crti.o -lc *.o -lcrypto -lssl -lsamplerate -lid3tag /usr/lib/x86_64-linux-gnu/crtn.o