Start early!
Choose the topic with both the assignment and the audience in mind. If the paper is a course assignment, the audience is usually the students in the class. For senior research, it is other math majors and faculty. Remember that some faculty may not be familiar with your particular problem, so be complete in your explanations. Design the paper with other aspects of the assignment in mind: page limit, goal (inform, entertain, teach, convince, ...), etc.
More than one reference is highly encouraged. Internet references are usually acceptable, but are often less reliable than print sources. The latter are generally refereed and have stood the test of time. Math books are in the QA section, upstairs south in the Stetson library. Math journals are alphabetical by title in the periodicals section, ground floor, north wing. Older journals are in the basement. Almost all of our journals are readable by motivated undergraduates.
The broad outline of your paper should be Title Page, Introduction, Major Topics, Conclusion, References. The paper should be organized into named sections to reflect this structure.
Develop your subject logically. Incorporate simple examples of all the important ideas, and illustrate with pictures or diagrams whenever possible. Don't forget "antiexamples," things that do not satisfy a definition.
An important rule is to make your paper easy to read. Normally, the reader will read in one or both of two fashions: a top-down skimming, and straight through for details. I do both. The top-down approach involves looking at the title and section names, possibly reading the first sentence in a paragraph to get the gist of the development. This means that the title and sections names should be accurate, short, and informative. This also means that any proofs should be well-organized, clear, and no wordier than necessary. A good paragraph proof leads the reader through it. See Section 2.1 of my book Essentials of Mathematics.
Use of good and correct writing is another important ingredient for a readable paper. Change paragraphs when you change the topic, vary sentence length and structure, check for spelling, grammar, and punctuation. Use the computer spell check, but read it through yourself also. Refer to your Brief Handbook for good writing style. But use the appropriate Math Referencing Style. More on that in the next paragraph.
Plagiarism can happen in math papers, just like in other subjects. However, mathematical ideas are far more likely to be considered in the public domain than are theories in the humanities or social sciences. For that reason, mathematical works contain few citations, and there is a distinctive style of referencing in mathematical papers. It is no less important to give credit where it is due, so please make an effort to learn the style. My handout Ethics in Mathematics Courses contains a discussion of what to reference and how.
Start with an outline of your paper. If the outline is to be submitted for comments, be explicit about what kind of thing and what detail will go into each section. Double-space the outline and leave wide margins, about 1¼ inches. Most word processors allow you to change the margins and line spacing easily.
Learn to use a mathematical typesetting program. Word has an equations editor, Mathematica has good typesetting abilities, as well as being a great calculation engine and producer of graphics if you need results like that in your paper. The professional standard for mathematical typesetting is TeX. The department is currently working on making TeX resources more available to students. Word is available on all the lab computers, and Mathematica is on the computers in 205E. Graphics from Mathematica can be pasted into Word and other documents.
Write a rough draft of the paper. If it will be submitted, make it as much like your idea of the final paper as possible. You want to get all the negative criticism you can at this stage☺ Again, double-space and leave wide margins.
The final paper should follow the guidelines given: length, formatting, font, margins, etc.