10/17/2019 1
1 1
State of Collaboration
Advance Your RFP Knowledge
2
- Introduction
- Nine RFP Case Studies
- Q&A
- Conclusion
Agenda
3
State of Collaboration Advance Your RFP Knowledge 1 1 Agenda - - PDF document
10/17/2019 State of Collaboration Advance Your RFP Knowledge 1 1 Agenda Introduction Nine RFP Case Studies Q&A Conclusion 2 CASE STUDY 1 CASE STUDY 1 3 1 10/17/2019 Solicitation Of Interest (SOI) What is
1 1
2
3
4
acceptance of offer.
5
6
evaluation and award.
participating any future formal procurement.
7
that can also provide these services.
– The SOI identifies required activities and minimum qualifications.
8
the services.
Request (CRER) to OSC including:
– Justification of vendor selection – Reasonableness of cost
9
vendors for these services and conducted a formal competitive procurement.
10
the most qualified.
– SOI is not a procurement vehicle and should not be evaluated. – SOI is not a formal procurement. When the potential for competition exists, the process shall not result in a contract. [SFL§163.7 Method of Procurement]
11
12
20% to most fixed price contracts because of uncertainty and risk caused by unclear specifications.
their distributed specifications to be of a high quality and easy to comprehend.
* Pettijohn, Carole and Qiao, Yuhua (2000). Procuring Technology: Issues Faced by Public Organizations, Journal of Public Budgeting, Accounting, and Financial Management 12 (2): 441‐461
13
Lowest required level of knowledge, ability or experience that a company as a whole, or any proposed staff, must possess to be considered for award of the RFP.
14
potential vendor.
abilities that your agency feels are “non‐negotiable”.
15
16
– Two (2) Pass/Fail Requirements – Three Evaluators
17
Vendor A’s Proposal
1
The Bidder must have successfully managed a point of sales system, for more than five (5) consecutive years and within the past eight (8) years. The contract for this engagement must be included as part of meeting requirement 2.
2
The Bidder must have managed a minimum of three (3) contracts, within the past eight (8) years. At least one of the contracts evidenced for this requirement must meet requirement 1. Point of Sale System Contract with NYC – 6 years
Evaluator 1 M 1 M 2 Vendor A ? ?
point of sale system five (5) consecutive years within the past eight (8) years three (3) contracts within the past eight (8) years.
18
Evaluator 1 Evaluator 2 Evaluator 3 M 1 M 2 M 1 M 2 M 1 M 2 Vendor 1 F F P F P P Vendor 2 P P P P P P Vendor 3 F F Vendor 4 P P P P P P Vendor 5 P F Vendor 6 P P P P P P
19
1. Vendor one (1) was evaluated by all three (3) evaluators, which agrees with the evaluation plan. However, the pass/fail conclusions do not agree among evaluators. 2. Vendors three (3) and five (5) were evaluated by one evaluator. The evaluation process did not follow the evaluation plan. 3. The mandatory requirements were confusing and possibly too restrictive.
20
21
– What you want to score in your RFP responses (desirables) – How you’re going to score the vendor’s responses (type of scale, weight) – Who is going to do the scoring (individuals or team)
22
evaluation of proposals.
requirement a mandatory.
23
Meets no aspects and/or expectations of the requirement or information is not present as required 1 Meets few aspects and/or expectations of the equipment; shows major gaps in knowledge and understanding of material; substantial inaccuracies 3 Meets most to all aspects and/or expectations of the requirement; shows understanding of key aspects and requirement 5 Exceeds aspects and/or expectations of the requirement; highly detailed knowledge and understanding of the requirement
24
# Requirement Mandatory or Desirable Score 1 The system must be capable of sending a confirmation to the customer’s designated preferred contact method. M 2 The system is capable of offering alternate options to customers in the event that the customer’s preference is unavailable. D 3 The system offers a mobile application(s) for authorized users and/or customers. D 4 Authorized users must be able to set and modify business rules. M 5 The system must have a process in place to ensure that profiles are unique. M 6 Contractor must manage customer logins and authentications. M 7 The system must allow for variations in fees based on business rules. M
25
requirement a mandatory.
the technical evaluation.
26
# Requirement Score 1 The system is capable of sending a confirmation to the customer’s designated preferred contact method. 2 The system is capable of offering alternate options to customers in the event that the customer’s preference is unavailable. 3 The system offers a mobile application(s) for authorized users and/or customers. 4 Authorized users are able to set and modify business rules. 5 The system has a process in place to ensure that profiles are unique. 6 Contractor manages customer logins and authentications. 7 The System allows for variations in Fees based on business rules.
27
28
“Where the basis for award is the best value offer, the state agency shall document……the determination of the evaluation criteria, which whenever possible, shall be quantifiable, and the process to be used in the determination of best value and the manner in which the evaluation process and selection shall be conducted.”
29
your RFP.
procurement.
evaluation instrument.
30
Component Maximum Points
Company Experience and Program Structure 15 Proposed Approach 30 Expansion of Training and Consultation Services 10 Personal Experience 10 References 5 Total Technical Points 70 Cost Evaluation 30 Total Proposal Score 100
31
Component Maximum Points
Company Experience and Program Structure 30 Proposed Approach 60 Expansion of Training and Consultation Services 15 Personal Experience 15 References 15 Total Technical Points 135 Cost Evaluation 30 Total Proposal Score 165
32
– (Proposer’s Raw Points ÷ Maximum Raw points) x Maximum weighted points = weighted technical score – Examples: Vendor A: (119 ÷ 135) x 70 = 61.70 Vendor B: (121 ÷ 135) x 70 = 62.74
70% Technical Weight 135 Maximum Raw Technical Points Vendor A Vendor B Proposer’s Raw Technical Points 119 121 Proposer’s Normalized Technical Points 61.70 62.74
33
Vendor Technical Score Cost Score Composite Score A 119.00 30.00 149.00 B 121.00 28.50 149.50 C 125.00 10.50 135.50 D 101.00 21.25 122.25
34
in the RFP.
points) which changed the Technical weight to 82%.
the weights specified in the RFP.
Evaluation Vendor A Vendor B Vendor C Vendor D Technical (70%) 61.70 62.74 64.81 52.37 Cost (30%) 30.00 28.50 10.50 21.25 Composite 91.70 91.24 75.31 73.62
35
36
– Rating Scale – Group discussion – Shortlisting – Interview/presentation
37
38
39
Vendor Name:_____________ Evaluator Name:_____________ Rating Scale 1 2 3 4 5 Signature:_____________ Date:_____________ Evaluation Criteria Max Points Initial Score Comments
5
5
5
40
Example of Irrelevant Comments:
Evaluation Criteria Max Points Initial Score Comments
5 1 Bad Handwriting
5 4
5 2 Poor Grammar
41
Example of Relevant Comments:
Evaluation Criteria Max Points Initial Score Comments
5 1 Only one manager listed, insufficient for all sites
5 4
5 2 Vendor unable to provide weekend staff
42
Scoring Matrix:
Evaluation Criteria Evaluator 1 Evaluator 2 Evaluator 3 Final Score Max Points
2 1 5 2.7 5
3 4 4 3.7 5
2 2 5 3 5
Total
9.4 15
43
Evaluation Criteria Max Points Initial Score Comments
5 5
5 4
5 5
Evaluation Criteria Max Points Initial Score Comments
5 1 Only one manager listed, insufficient for all sites
5 4
5 2 Did not mention coverage for time off Evaluation Criteria Max Points Initial Score Comments
5 2 Inadequate staff for multiple sites
5 3
5 2 One of four preferred qualifications met
44
45
“Methods for calculating costs vary depending on a mix of factors concerning the nature and extent of the services, the costs associated with utilizing the services, and the impact of the services on agency programs and operations (State Finance Law§160.5 and 6).”
46
– Costs submitted must be completed in accordance with RFP. Do not accept narratives.
47
based on the proposed cost.
appears to be out of line with the other proposals.
48
Cost Score = (Lowest Bid ÷ Bid Being Evaluated) X Maximum Cost Points Available
49
50
Job Title Hourly Rate Estimated Hours Total Cost
Title 1 To be entered by vendor To be entered by agency $ Title 2 To be entered by vendor To be entered by agency $ Title 3 To be entered by vendor To be entered by agency $ Title 4 To be entered by vendor To be entered by agency $
Total Bid Amount $
51
evaluation purposes.
Non‐recommended options for evaluating hourly rates:
Job Title Hourly Rate
Manager $160
$90 Analyst $55 Tech Staff $45
$87.50
Job Title Hourly Rate
Manager $130
$110 Analyst $65 Tech Staff $55
$90
52
Recommended options for evaluating hourly rates:
Job Title Hourly Rate # of Hours Total Manager $160 3,000
$90 1,000 Analyst $55 2,000 Tech Staff $45 4,000 Total Job Title Hourly Rate # of Hours Total Manager $130 3,000
$110 1,000 Analyst $65 2,000 Tech Staff $55 4,000 Total $480,000 $90,000 $110,000 $180,000 $860,000 $390,000 $110,000 $130,000 $220,000 $850,000
53
– Cost will be based on hourly rates. – Cost sheet requires vendors to provide a rate for each title listed below. – Two (2) vendors submitted proposals. – Vendor A was deemed non responsive.
Job Title Rates # of Hours Manager $140 ‐ $160 3,000
$80 ‐ $90 1,000 Analyst $55 ‐ $75 2,000 Tech Staff $40 ‐ $60 4,000 Job Title Rates # of Hours Manager $150 3,000
$75 1,000 Analyst $55 2,000 Tech Staff $65 4,000
54
required for this contract.
Title Rate Weight Points
Project Manager $300 70% 21 Programmer $175 20% 6 Support Staff $100 10% 3
Total 100% 30
55
56
57
Vendors could propose either a % of compensation with a minimum fee OR vendors could propose a flat fee.
PROFESSIONAL FEES % of Compensation % Minimum Fee $ Flat Fee $
58
Vendor A Vendor B Vendor C PROFESSIONAL FEES Pricing Score Pricing Score Pricing Score % of Compensation (10 pts) 30% 9.33 28% 1.00 Minimum Fee (10 pts) $45,000 10.00 $72,000 16.25 Flat Fee (20 pts) $60,000 20.00 TOTAL COST SCORE 19.33 20.00 16.25
59
60
“The evaluation criteria and methodology for evaluating proposals must be completed prior to the initial receipt of proposals.”
61
in the evaluation:
interview/oral presentation. If this method is used, evaluators must address any changes to the original scores.
62
in the evaluation (continued):
evaluators ask the same thing of all vendors.
proposals are received.
63
through the final steps of the evaluation process.
cost scores. – Shortlisting without considering cost is arguably inconsistent with the SFL “best value” concept.
64
necessary by an Agency.
plans to utilize this technique.
the preliminary technical and cost scores. – Example: If the oral presentation and interview is worth 10 points, anyone within 10 points of the highest scoring vendor is susceptible to award and must be shortlisted.
65
66
Vendor Technical Score Cost Score Composite Score A 53.33 19.33 72.66 B 54.00 20.00 74.00 C 42.97 16.25 59.22 D 59.00 16.70 75.70 E 62.00 11.96 73.96 F 56.00 16.76 72.76 Vendor Technical Score Cost Score Composite Score D 59.00 16.70 75.70 B 54.00 20.00 74.00 E 62.00 11.96 73.96 F 56.00 16.76 72.76
The proposal that receives the highest composite score and the next three highest scoring proposals, if they are all within 10 points of the highest total score, will be shortlisted and invited for an oral presentation.
67
the results were as follows:
Vendor Technical Score Cost Score Oral Presentation Composite Score D 59.00 16.70 6.50 82.20 B 54.00 20.00 8.00 82.00 E 62.00 11.96 7.00 80.96 F 56.00 16.76 9.00 81.76
This is incorrect.
68
Vendor Technical Score Cost Score Composite Score A 53.33 19.33 72.66 B 54.00 20.00 74.00 C 42.97 16.25 59.22 D 59.00 16.70 75.70 E 62.00 11.96 73.96 F 56.00 16.76 72.76
were shortlisted.
A and D have the highest composite scores and should be awarded contracts.
Vendor Technical Score Cost Score Composite Score Oral Presentation Total Score A 53.33 19.33 72.66 10.00 82.66 B 54.00 20.00 74.00 8.00 82.00 D 59.00 16.70 75.70 6.50 82.20 E 62.00 11.96 73.96 7.00 80.96 F 56.00 16.76 72.76 9.00 81.76
69
70
understanding of responsiveness to the solicitation requirements
[SFL 163.9(c)]
71
72
73
– Vendor 1: “I have experience in JavaScript” – Vendor 2: “I have experience in the required program” – Vendor 3: “I have experience in programs that are similar to JavaScript”
74
the requirements
– Ask Vendor 2 and Vendor 3 if they have experience in JavaScript – Give the vendors a chance to further explain
record
75
Vendors were required to provide the following as part of their cost:
» Job Title A » Job Title B » Job Title C
76
Required Vendor A Vendor B Vendor C
Job Title A $100 $ $100 Job Title B $80 $80 $80 Job Title C $60 $60 $60 Estimated 2,080 Hours for Each Job Title $499,200 $291,200 $499,200 One Time Start‐Up Fee $ $3,000 $300,000 Annual Maintenance Fee $4,500 $4,500 $4,000 Total $503,700 $298,700 $803,200
77
Required Vendor A Vendor B Vendor C
Job Title A $100 $80 $100 Job Title B $80 $80 $80 Job Title C $60 $60 $60 Estimated 2,080 Hours for Each Job Title $499,200 $457,600 $499,200 One Time Start‐Up Fee $0 $3,000 $300 Annual Maintenance Fee $4,500 $4,500 $4,000 Total $503,700 $465,100 $503,500
78
79
80