I am not an expert. Much of what I have learned has been through trial and error. In preparation for my first teaching gig, I did study quite a bit about color theory and I continue to study, and I have learned that there is not a general consensus on even the most basic concept - that of primary colors.
Hang with me here because this is a whirlwind of information.
I have recently bought my first good, artist grade brushes. The bristles are natural (as opposed to synthetic) sable. They were expensive but worth every penny. Natural hairs hold much more water and pigment than synthetic, which reduces the number of times you will need to reload your brush. Sometimes, constantly needing to reload your brush (dipping it into water and pigment) can result in a streaky painting, especially if you are doing a big wash (filling up the page), such as you would for painting a landscape or sky. You can get away with cheaper brushes for small, detailed work, which is probably where you will start. At some point, if you stick with it long enough and enjoy it, I would eventually invest in good brushes, but for now, you can go with student grade which is what you will mostly find at the craft stores. You can use as little as 2 brushes - a size 4 filbert and a size 2 round - or you could use a water brush if all you will be doing is working in your nature journal. These are wonderful for working in the field. The water is stored in the barrel of the pen so there is no need to use a cup of water for wetting and cleaning the brush. I recommend the Pentel Aquash brand. Cheaper ones can leak, allowing pigment to enter the barrel.
I recently began using good artist grade paper. It is delicious. I love, love, love working with it. The paper does not buckle as badly (Wet a piece of cheap watercolor paper, and let it dry. The result is buckling.) It is also very forgiving. I am able to scrub off color without damaging the paper. That being said, it is not necessary to begin with good paper, especially if you will be doing small, detailed work which involves less water. The more water you will be using, the heavier weight paper you will need (paper's come in weight - 300 weight does not buckle.)
You will need to use watercolor paper though. Do not try to get by with card stock or copy or drawing paper. You will abhor the results. I started with Canson. It is cheap and can be bought in large pads. The paper can then be torn down to make smaller sheets. You can even bind the sheets together to make your own journal. Here is a link to bookbinding tutorial on Youtube.
And here is where it begins to get complicated. As I said before, there is no consensus on even the basics. Artists disagree on even the primary colors and hence on what you should use in your basic palette. Many artists believe that the three primary colors are cyan, magenta, and yellow. I could go into this, and I did at my first teaching gig, but because I am not going to stick with this, I will leave it to you to research. If you mix magenta and cyan together you will get a lovely, clear shade of purple. If you mix cyan and yellow together, the shade of green will be jewel like. But I have found that I very rarely need these jewel tones when nature journaling, so I prefer to stick with a more traditional palette. If I had to paint with only three colors they would be Alizarin Crimson, French Ultramarine or Pthalo Blue, and Quinacridone Gold with Naples Yellow being a close alternative. These are the workhorses of my palette, the colors I run out of most often.
With just three colors you could mix anything you would need. But that does get kind of tiresome. So I would include a brown - Burnt Umber as this is often needed in nature journaling. I very rarely paint with just one color. Meaning even for brown, I will usually mix another color into it. Sometimes I will see the brown tending towards blue shade, so I will mix in some of my French Ultramarine. Sometimes I will see the brown with hints of red, so I will mix in my Alizarin Crimson.
So there you have four colors - not undoable budgetwise.
French Ultramarine (or Pthalo)
Quinacridone Gold (or Naples or Hansa light).
I have one more favorite that is not a necessity but I love it nonetheless and use it often - Bloodstone Genuine. I use this one a lot for shadows, especially effective when painting sand or soil. Bloodstone Genuine has high granulation - which means that when the paint dries it will look grainy. I used Bloodstone for the shadows in this painting of a streambed.
Not all brands have the same name. For example, Winsor and Newton's Winsor Blue contains the same pigment as Daniel Smith's Pthalo Blue. The way to tell if it is the same is to find the paint's signature number on the side of the tube.
Whew! I think that's it for now.