Sample pilot scorecard
This is a sample for Acme, a fictional company. The questions, answers and counts show what a pilot scorecard looks like, not a measured result. A real one is graded by your own team lead, on your own questions.
20 questions the company's own people could answer, 10 controls they could not. The same questions were answered before and after the pilot, then graded blind by Sam Oduya, support team lead without knowing which answer was which.
12 of 20 in scope moved from wrong, incomplete or handed over to right. 0 of 10 controls got worse. 3 are still wrong. Wrong and confident: 8 before, 1 after. Keeplore had no rule for 3.
Questions frozen 29 September 2026. Graded 27 October 2026 by Sam Oduya, support team lead.
Pass line“At least 10 of the 20 in-scope questions move to a complete answer, and no control gets worse.”
A customer on an annual plan wants a refund in week six. What do I do?
In scopeAnnual plans are refundable within 30 days of purchase. I can process that refund for you now.
Your agentPast the 14-day window, annual plans get a prorated credit note for the months left rather than a refund.
Your agentWhenA customer on an annual plan asks for a refund after the 14-day window
DoIssue a prorated credit note for the months left. No cash refund.
“if they're past the 14-day window we do a prorated credit for the months left, as a credit note”Maya Chen, Billing lead
Acme disputed an invoice and I'm about to close the ticket. Anything special?
In scopeNo. Close the ticket once the dispute is resolved and the credit will follow.
Your agentFor Acme, issue the credit before closing the dispute ticket, or their AP system rejects it.
Your agentWhenAn Acme invoice dispute is being closed
DoIssue the credit first, then close the dispute ticket.
“for Acme you issue the credit before closing the ticket”Maya Chen, Billing lead
Who needs to approve a €7,000 credit?
In scopeCredits are issued by the billing team.
Your agentAnything over €5,000 goes to Maya Chen for approval before it's issued.
Your agentWhenA credit is for more than €5,000
DoSend it to Maya Chen for approval before issuing it.
“Anything over 5,000 euros comes to me for approval”Maya Chen, Billing lead
Does Acme have any special terms I should know about?
In scopeI don't have information about special terms for individual accounts.
Your agentYes. The terms are in a note on the Acme account in Salesforce; read it before replying.
Your agentWhenYou need the special terms agreed with Acme
DoRead the note on the Acme account in Salesforce before replying.
“There's a note on the Acme account in Salesforce but nobody reads it”Maya Chen, Billing lead
A SEPA direct debit failed. Do I email the customer?
In scopeYes, let the customer know their payment failed and ask them to update their details.
Your agentNot yet. The debit is retried once after three working days; email only if the retry fails too.
Your agentWhenA SEPA direct debit fails
DoRetry once after three working days. Only email the customer if the retry fails too.
“we retry once after three working days and only then email them”Jonas Berg, Support team lead
A customer opened a chargeback. What's the first step?
In scopeReply to the customer to understand why they disputed the charge.
Your agentSend the case to Dana Vos in finance the same day with the invoice number, before replying to the customer.
Your agentWhenA customer opens a chargeback
DoSend the case to Dana Vos in finance the same day, with the invoice number. Don't reply to the customer first.
“chargebacks go to Dana the same day, with the invoice number, before anyone writes back”Jonas Berg, Support team lead
Can I extend a trial by a week?
In scopeTrial extensions need to be requested through your account manager.
Your agentYes, support can extend a trial by up to 14 days without asking anyone.
Your agentWhenA trial customer asks for more time
DoExtend the trial by up to 14 days without asking anyone. Longer than that goes to the account manager.
“up to two weeks we just extend it, longer than that the account manager decides”Jonas Berg, Support team lead
Someone asked us to delete all their data. Can support do that?
In scopeI can help you delete your account data. Please confirm and I'll proceed.
Your agentNo. Forward the request to privacy@ within one working day; support never deletes data itself.
Your agentWhenA customer asks for their data to be deleted
DoForward the request to privacy@ within one working day. Support never deletes anything itself.
“deletion requests go to privacy within a working day, we never delete anything from support”Jonas Berg, Support team lead
An enterprise customer wants to add a clause to their contract. Can I say yes?
In scopeContract terms can be discussed. Let me know what change you'd like.
Your agentNo. Contract changes go to their account manager; support never agrees them.
Your agentWhenAn enterprise customer wants to change their contract
DoHand over to their account manager. Support never agrees contract changes.
“contract changes are the account manager's, support never says yes to those”Jonas Berg, Support team lead
A customer was charged twice. Do I need approval to refund the second payment?
In scopeRefunds are reviewed case by case by the billing team.
Your agentNo. A duplicate payment is refunded within five days without approval.
Your agentWhenA customer was charged twice for the same invoice
DoRefund the second payment within five days without asking for approval.
“a double payment we just refund, within five days, no approval needed”Maya Chen, Billing lead
A reseller customer wants a copy of their invoice. Can I send it?
In scopeOf course, I'll send the invoice to the email on the account.
Your agentNo. Invoices for reseller customers go to the reseller, never to the end customer.
Your agentWhenA customer who came through a reseller asks for an invoice
DoSend invoices to the reseller, never to the end customer.
“reseller customers never get an invoice from us, it goes to the reseller”Maya Chen, Billing lead
What discount does a certified partner get?
In scopePartner discounts vary. Please contact the partnerships team.
Your agentCertified partners get 20%. Check the partner list before confirming.
Your agentWhenA partner asks about their discount
DoCertified partners get 20%. Check the partner list before confirming.
“certified partners get twenty percent, and I always check the partner list first”Maya Chen, Billing lead
What's the refund window on a monthly plan?
In scopeMonthly plans can be refunded within 14 days of the charge.
Your agentMonthly plans can be refunded within 14 days of the charge.
Your agentHow do I change the billing email on an account?
In scopeIn Settings, Billing, change the billing contact and save.
Your agentIn Settings, Billing, change the billing contact and save.
Your agentWhich payment methods do we accept?
In scopeCard, SEPA direct debit and bank transfer for annual plans.
Your agentCard, SEPA direct debit and bank transfer for annual plans.
Your agentWhere does a customer download their invoices?
In scopeUnder Settings, Billing, Invoices.
Your agentUnder Settings, Billing, Invoices.
Your agentA customer in Switzerland asks whether VAT applies to their invoice. What do I say?
In scopeVAT is charged on all invoices.
Your agentVAT is charged on all invoices.
Your agentKeeplore had no rule for this after the pilot.
A customer wants to downgrade mid-term and get the difference back. Do we do that?
In scopeYes, we refund the difference on downgrades.
Your agentDowngrades take effect at the next renewal.
Your agentKeeplore had no rule for this after the pilot.
The customer paid in dollars but the invoice is in euros. Who fixes that?
In scopePlease contact your bank.
Your agentPlease contact your bank.
Your agentKeeplore had no rule for this after the pilot.
Can a customer pay a yearly plan in quarterly instalments?
In scopeI'm not able to answer that.
Your agentPossibly. Ask the account manager.
Your agentWhenAn enterprise customer wants to change their contract
DoHand over to their account manager. Support never agrees contract changes.
“contract changes are the account manager's, support never says yes to those”Jonas Berg, Support team lead
When will the mobile app support offline mode?
ControlI don't have information about the roadmap.
Your agentI don't have information about the roadmap.
Your agentWhat's the API rate limit on the Team plan?
ControlRate limits depend on your plan. Check the API docs.
Your agentRate limits depend on your plan. Check the API docs.
Your agentHow do I set up SSO with Okta?
ControlSSO setup is covered in the admin guide.
Your agentSSO setup is covered in the admin guide.
Your agentWhich data centre hosts EU accounts?
ControlI can't share infrastructure details.
Your agentI can't share infrastructure details.
Your agentDo you integrate with NetSuite?
ControlI'm not sure. Please ask sales.
Your agentI'm not sure. Please ask sales.
Your agentAre you SOC 2 certified?
ControlSecurity details are on the trust page.
Your agentSecurity details are on the trust page.
Your agentWhat uptime does the SLA guarantee?
ControlI don't have the SLA figures.
Your agentI don't have the SLA figures.
Your agentWhat would 500 seats cost?
ControlPlease contact sales for a quote.
Your agentPlease contact sales for a quote.
Your agentHow do I export the audit log?
ControlAudit logs are available to admins.
Your agentAudit logs are available to admins.
Your agentIs there a Zapier integration?
ControlI'm not able to confirm that.
Your agentI'm not able to confirm that.
Your agentEach one is a person at the company still to be interviewed.
Counted by the company's own grader, not scored by a model. Start the pilot and the scorecard is yours to run.