First off, copy your handwritten notes to the computer. Then, when you start to organize things you won't have a blank page staring at you (and I know how frightening a big white space can be). Then, when an idea hits you, you can move or copy/paste the pertinent passage(s) to your working document and develop the chapters from there.
Took me three weeks to write my first book, took me 3 years to make it readable, and I'm still seeking a publisher. I sent the first 10K words to an agent; still waiting for a response.
