Collegiate Reviewer Instructions
Thank you for helping to review the NCWIT Collegiate Award applications! The applicants are undergraduate and graduate students who self-identify as women, genderqueer, or non-binary in any year of college and are majoring in computing and related technical fields. We are looking for applicants with outstanding technical accomplishments, demonstrated by projects that showcase their skill, mastery of the subject and expertise. Higher scores should be awarded to more innovative, impactful and difficult technical projects where the applicant was able to define their role clearly and confidently.
Please keep the following important guidelines in mind when scoring each application:
Applicants are broken into three categories:
- freshman, sophomore or two-year degree program
- junior or senior
- graduate student
- Applicants will, and should, be at different levels of technical experience and expertise. Please take the “grade level” into consideration when viewing the application materials, scoring the technical questions and reviewing the applicant as a whole. For example, a Ph.D. applicant should typically display a high degree of independence and technical savvy in comparison to a typical Freshman/Sophomore application. Score them against their peer-group expectations, to the extent possible.
- Applicants may not have English as their first or native language. Please do not penalize such applicants for minor syntax errors.
- Applicants have been asked to explain why their technical project is innovative or impactful. We urge you to put evidence of technical ability at the heart of your scoring. Please attempt to use your best understanding of the technology addressed to assess “how hard” it was to attain their accomplishment - rather than how well they presented it, or how “cool” it is. A project contribution that uses technology to solve a problem in a whole new way ought to be scored higher than something socially-positive or cute, but not that difficult to accomplish (technically).
- If you feel like an application contains a level of technical depth that exceeds your knowledge in the subject area (remember, some are Ph.D. candidates), please feel free to abandon your review and leave it to others. To do so, simply leave the Review page without saving the review (i.e., go “Back” in your browser), do not select the “Discard Review” button.
- You may be assigned an application for a student you personally know, or who attends an institution to which you have personal connection. If you think that your relationship with the applicant or her college/university presents a conflict of interest, please do not complete your review of the application. Simply leave the Review page without saving or submitting the review (i.e., go “Back” in your browser), do not select the “Discard Review” button.
APPLICATION REVIEW QUESTIONS
Project Description
Applicant Prompt: Please provide a thorough description of the project. Explain who your project is designed to benefit/impact. Explain the problem the project solved, and how, without requiring the reader to be an expert with deep background knowledge.
Reviewer Prompt: Please utilize the point descriptors below to score the Project Description.
1. The application materials as a whole (including all written essays and presentation video) DO NOT clearly state the PROBLEM SOLVED nor why it is important to solve. Or the problem solved, and how it is solved, is not explained. The reader cannot glean the essential scope and details of the project.
2. The application materials as a whole (including all written essays and presentation video) state the PROBLEM SOLVED or why it is important to solve but are LACKING IN DETAILS on either the problem or the solution. The reader has a limited understanding of the scope and details of the project.
3. The application materials as a whole (including all written essays and presentation video) state the PROBLEM area, the importance of the problem area, and give a general picture of the applicant's work to solve the problem. The applicant's solution and work is EXPLAINED WITH SOME DETAIL.
4. The application materials as a whole (including all written essays and presentation video) CLEARLY state the PROBLEM area, the importance of the problem area, and describe the applicant's work to solve the problem with some detail. The applicant's solution and work is explained with clarity and depth.
5.The application materials as a whole (including all written essays and presentation video) CLEARLY state the PROBLEM area, the importance of the problem area, provide relevant contextual information, and paint a clear picture of the applicant's work to solve the problem. The applicant's solution and work is explained with significant detail. The reader can clearly grasp the scope and details of the project.
Project Interest/Innovation
Applicant prompt: Whether you chose the project or were assigned to it, please describe the aspects that were innovative and interesting.
Reviewer Prompt: Please utilize the point descriptors below to score this essay.
1. The project lacks any aspects that are innovative or interesting.
2. The project has a LIMITED amount of innovation and interest.
3. The project has a MODERATE amount of innovation and some interesting aspects.
4. The project is CLEARLY innovative and interesting.
5. The project has REAL INNOVATION and has a high number of interesting aspects.
Technical Complexity
Applicant Prompt: Clearly describe each of the technical skills that you utilized in the creation of this project and explain why each skill was vital to the success of the project.
Reviewer Prompt: Rate the technical complexity of the project based on technical skills used, along with overall description of the project.
1. The project has NO CLEAR technical complexity or difficulty. The skills described are very basic in nature.
2. The project has a relatively LOW LEVEL of technical complexity or difficulty. The skills described are a mix of basic skills and intermediate skills.
3. The project has a MODERATE LEVEL of technical complexity or difficulty. The skills described are intermediate skills.
4. The project has a HIGH LEVEL of technical complexity or difficulty. The skills described are a mix of intermediate and advanced skills.
5. The project has an EXTREMELY HIGH LEVEL of technical complexity or difficulty. The skills described are highly advanced skills.
Project Contribution
Applicant Prompt: Referencing your answers above (in the multiple choice section), please describe your individual contribution to this project.
You – the reviewer – may refer to these multiple choice items to view the student’s self-assessed contribution:
- How did the idea for the project come about? (assigned, out of necessity, own interest...)
- How much of the overall work was your contribution?
- How significant was your contribution to the project outcomes?
- Considering the portion of the project that you worked on, how much supervision and direction were you given?
Reviewer Prompt: The suggested score is based on the multiple choice questions under "Project Contribution " and the applicant's response to the Personal Contribution written response question. Pay particular attention to the technical innovation and leadership. In some cases a student may have a non-technical role that is equally as significant as the technical contributions. In addition, note that students who make a contribution to a group project may be equivalent to a sole contributor on an individual project, if their role was extremely impactful to the project. Utilize the point descriptors below.
1. Student had ALMOST NO contribution to the project.
2. Student had MINOR contributions to the project.
3. Student had MODERATE contributions to the project.
4. Student had MAJOR contributions to the project.
5. Student was the SOLE CONTRIBUTOR to the project.
Project Presentation
Applicant Prompt: Please upload a presentation detailing your project (length should be 4-6 minutes, with an absolute maximum of 6 minutes). The presentation must be an original work created for the NCWIT Collegiate Award. This presentation may be a narrated powerpoint presentation, a screencast demonstrating your work, or a multimedia presentation. In order to preserve anonymity during the review process, we ask that you do not include personally identifying information (e.g. your name or your school) and that you do not appear on screen in the video.
Reviewer Prompt: Please utilize the point descriptors below to score the clarity and effectiveness of the presentation. Since applicants have been given a maximum of 6 minutes for their presentations, score the presentation on the first 6 minutes if it is longer.
1. The presentation was not very effective or organized and the description of the project was inadequate.
2. The presentation was moderately effective/organized and conveyed a moderately complete description of the project.
3. The presentation was fairly well organized and effective and conveyed many of the elements of the project.
4. The presentation was highly organized, effective, and conveyed the message of the project appropriately.
5. The presentation demonstrated an extremely high level of organization and effectiveness. The message of the project was fully and completely conveyed in a highly professional manner.
Recommendation
Applicant Prompt: To aid our application reviewers, we’ll ask you to share with us the contact information for someone who knows many of the details about your project and can write a complete explanation recommending you and your project for its technical quality, difficulty and potential impact. We need a name and email address for this recommender who is likely a Professor (but need not be). We will contact that person and ask them to submit a recommendation about you, your project and its level of achievement versus your peers.
Reviewer Prompt: Please utilize the point descriptors below to score the recommendation. Note that recommendations should not include identifying information.
1. The recommendation lacked elements that would have clarified support.
2. The recommendation was fair and contained necessary elements.
3. The recommendation was very supportive and contained clear details.
4. The recommendation was highly detailed and clearly explained why the recommender supports the applicant.
5. The recommendation showed exceptional support of applicant, citing many examples of applicant's talent and skill.
Summarization of Student
Additional Applicant Prompts:
- What are the roadblocks that stand in your path to success?
- What supports in your life help make your work possible?
- What impediments do you have in your life that limit your work?
Reviewer Prompt: Considering this application in its entirety, please indicate your overall recommendation for this applicant. Recommended applicants may be invited to participate in the Finalist round for the NCWIT Collegiate Award.
1. Application is VERY WEAK and SHOULD NOT BE CONSIDERED AT ALL for the Finalist Round of the NCWIT Collegiate Award.
2. Application is WEAK and SHOULD NOT MOVE FORWARD to the Finalist Round of the NCWIT Collegiate Award.
3. Application is ADEQUATE and SHOULD MOVE FORWARD WITH RESERVATIONS to the Finalist Rround of the NCWIT Collegiate Award.
4. Application is STRONG and SHOULD MOVE FORWARD to the Finalist Round of the NCWIT Collegiate Award.
5. Application is EXTREMELY STRONG and SHOULD MOVE FORWARD WITHOUT RESERVATION to the Finalist Round of the NCWIT Collegiate Award.
*Reviewer Comments: Reviewer comments are required. Your comments are invaluable and make a big difference during the selection phase.
