In order to be able to build an embedded Linux system, obviously you need:
The GNU cross-development tools installed on your host machine (presumably a Linux PC machine).
Your target system will need to have these different pieces of software:
Firmware (or boot loader), such as u-boot, redboot or a BIOS: you can these choose to boot from Flash, network (tftp), or disk;
Kernel (linux): the kernel must be compiled with support for your board (cpu and peripherals);
Root file system (contains the C/C++ libraries, the shell, etc.).
If your board comes with a ‘good’ firmware, you might not need at all a JTAG programmer. Using the firmware, you might be able to program the flash with the kernel and/or the root filesystem. Both u-boot and redboot let you do this, assuming they have the drive for the Flash you are using.