The Zombie Stack Exchanges That Just Won't Die
If a library is looking into forming a digitization team dedicated to digitizing your rare & special collections:
jambina
I came across an interesting set of resources for digitization today. It's called Digital Preservation in a Box, created by the Library of Congress's National Digital Stewardship Alliance (NDSA). Resources include tools and discussions of format and risk issues. It's too much to really encapsulate here, but it does address some of your questions. The Library of Congress Standards are on their website, but may be more technical than what you are looking for.
An additional, exhaustive set of online resources can be found on the Digital Curation Resource Guide. The list includes links to discussion groups, file formats and guidelines, sample library policies, standards, software and tools, and blogs. The list was compiled by Charles W. Bailey, Jr., formerly Assistant Dean for Digital Library Planning and Development at the University of Houston Libraries.
The LIFE-SHARE Project produced a useful set of resources that help to take the reader through the process of establishing and implementing a digitisation activity, as well as guiding dissemination and preservation (See the Digitisation Toolkit). There is also an end to end case study on digitising Special Collections which made these recommendations (summarised to keep this response concise):
The project also explored consortial models, how they might be applied in order to create shared digitisation services, and what aspects of interaction were important in order to create effective partnerships (trust, communication, coordination, etc).
3.Is there a standard that should be followed for preservation?
An ALA task force is documenting the minimum recommended tech specs for digitization. It's only in draft now, but it's well-cited and concise. http://connect.ala.org/node/185648
Guidelines on digitization for various materials (still images, moving images, audio...) can be found at the Federal Agencies Digitization Guidelines site: http://www.digitizationguidelines.gov/. Their project planning materials http://www.digitizationguidelines.gov/guidelines/digitize-planning.html may be of some use in exploring how to tackle digitization efforts. While your organization may make some different choices, these guidelines should be a useful starting point for your organizational discussion.
Unpacking the first part of the first question....
How should what gets digitized be determined?
Creating a policy to prioritize or queue items for digitization is nearly impossible to perfect. That's an anecdotal observation from working on digitizing special collections projects for two years as an academic librarian at 75% time with digital curation and 25% time with special collections. The primary reason is priorities will change on a near daily basis.
Factors influencing (remixed from NEDCC, JISC, NDSA, LOC, & RBMS among others) the development of a prioritization policy could include (I won't say they are ordered, as that would put into question the veracity of my already anecdotal answer):
I'd like to add some small advice on Paul Wheatley's answer regarding file formats. Archival master images should always be saved as uncompressed files, as compression comes with some problems. If i. e. just one bit in the compressed image flips, the rest of your master (behind that bit in the bitstream) is useless and lost. Also, compression uses up cycles on your archival servers that will be better used doing other things like creating downscaled copies of your master images for presentation.