GSL - The GSL is a good general purpose library of numerical routines written in C. It has many of the same algorithms you will find in Numerical Recipes. I tend to use this for anything that needs to be compiled, but can still be run serially.
CLAWPACK - CLAWPACK is a Fortran based library for solving systems of conservation laws. The authors have recently begun migrating it to Fortran 90/95 and implemented Python for plotting and batching jobs.
I have used CLAWPACK to produce these animations of the primitive nonlinear Maxwell equation. For this work, I solved the Riemann problem exactly, using the GSL, as it is nonconvex. This required some nice mixing of C and Fortran.
PETSc- PETSc is a numerical linear algebra library written in C intended for solving PDEs. It is designed to work with MPI with the intention of being used on clusters.
FEniCS - FEniCS is a library for solving FEM discretized PDEs. It automates much of the code generation, but is still a work in progress.
I used FEniCS, with PETSC as the linear algebra backend and MPI, to solve ensembles of Stokes problems, such as these.
SciPy - SciPy is a Python based library of numerical routines, many of them built on old Fortran libraries, like MINPACK. I now find that SciPy, together with NumPy, and iPython notebooks is, in many ways, superior to MATLAB.
I used SciPy for the computations in my publications with J.L. Marzuola and S. Raynor on saturated nonlinearities, and with M. Spiegelman for the solitary wave benchmarks. This was particularly useful in the solitary wave benchmark research, as SciPy easily integrates with FEniCS through Python.
BVP_SOLVER - This is Fortran 90/95 based routine for solving two point boundary value problems with singularities.
This software was used in my work with J.L. Marzuola and R. Asad for ruling out embedded eigenvalues in the linearized NLS problem, and in studying blowup of vortex solitons in my paper with I. Zwiers.
There is a Python scikit based on it available at BVP_SOLVER scikit
TorontoMathWiki - I have been actively adding material to the Toronto Math Wiki geared towards performing simualtions in MATLAB. Much of my content is geared towards solving problems pseudospectrally, but there are also suggestions for plotting and running batch jobs. See Category:MATLAB for some of my entries.
Though I have a few suggestions of my own, my favorite resource for LaTeX information is the LaTeX Wikibook
Use the AMS packages: amsmath, amsfonts, amssymb. In particular, use \begin{align}...\end{align} in place of \begin{eqnarray}...\end{eqnarray} for grouping equations; I get better results with less work. Also, for a single equation that must be broken up across multiple lines, use \begin{split}...\end{split} within a \begin{equation}...\end{equation} environment.
For drafting up manuscripts and notes, use the amsart document class. It is in many ways superior to the standard latex article class.
Use the package showkeys to have your the labels you assign equations appear while working in draft mode.
Use the package cite to have references appear as [1--5] instead of [1,2,3,4,5].
Use the package hyperref. Without doing anything further, PDFs will now pick up the sections of your document.