I need some expert review this test plan document and add your comments or update existed document, and this test plan is between two companies accordingly.
OregonBuys Quarterly Release and Yearly Upgrade Test Plan Version: 1.0 Date: 1/19/2021 Any locally downloaded versions may not be up to date. Document Information Document Title Testing Plan – OregonBuysStatefirst Quarterly Release and Yearly UpgradeComment by Hornig, Tim: Was v1.0 of this plan tied to v15.0 of Statefirst? Version information here and below is not clear…Comment by HARRIS DeAnna * DAS: I updated the wording, it isn’t for a specific version. Version 1.0 Author De Anna Harris – OregonBuysStatefirst Business Analyst Owner De Anna Harris – OregonBuysStatefirst Business Analyst Version Control Date Version Description Author 1/19/2021Comment by Hornig, Tim: Hasn’t this been updated since January? Comment by HARRIS DeAnna * DAS: I will go back and look at the changes I made and update this. 1.0 Draft De Anna Harris Contents Document Information1 Version Control1 Objective3 Scope3 Schedule3 Resources5 Interface Testing Process5 Interface Testing Control Procedures6 Reconciler Module6 Outcome documentation7 Issue/Question Review Meetings7 Defect Management and Resolution7 Approval of testing results8 Glossary Agency – official departments, divisions, commissions, boards, and agencies of the government of the State of Oregon. Control Report- Transaction Report from OregonBuysStatefirst DAS – Department of Administrative Services, owner of OregonBuysStatefirst. DAFR Report – Transaction Reports from R*STARS Defect - a reproducible failure of the Solution to operate in accordance with the Solution Requirements stated in Exhibit A-1 and specifications in the WOC, Statement of Work, despite the proper use of the Solution. A Defect may be due to an error caused by failures in data conversion or failures caused by the Solution Processing, or a problem with the Solution, or both. Prior to acceptance, a Defect may be due to a missed, missing, or misinterpreted Solution Requirement. Department – An organizational hierarchy level in OregonBuysStatefirst. A sub-element of Organization representing lower hierarchical structure within an agency. OBE – OregonBuysStatefirst Enterprise. OBE Project Team – The State’s core project team. OBE Testing Team- Agency users that have agreed to assist in quarterly testing. Patch – an update what resolves security vulnerabilities and/or critical defects to the Solution or SaaS application. Patches are issues against Upgrades and Releases and are represented by a suffix to a Release number (example: move from 15.4 to v15.4.1) PHI – Periscope Holdings, Inc. Reconciler – OregonBuysStatefirst functional module to be used for the purposes of VSR reporting. Release – updates that do not include substantial changes to the Solution or SaaS application and are represented by minor versioning numbers (example: move from 15.4 to 15.5). Releases may include both enhancements to modules and defect resolution. RSTARS – R*STARS, Relational Statewide Accounting and Reporting System. SLA – Service Level Agreement, generally specified in Appendix F of the OregonBuysStatefirst Price Agreement. SMEs – Subject Matter Experts, typically involved in collaborative meetings on “As Is” processes and systems and “To Be” prototyping. Solution – the end-to-end eProcurement system provided through the SaaS application that is comprised of the Deliverables and Work Product, Contractor Intellectual property, Third party intellectual property, Software, Hardware, Interfaces, and Documentation SPOC – Single Point of Contact. UAT – User Acceptance Testing environment. Upgrade – upgrades, improvements or changes to the specific platform or the Solution that are generally made available by PHI at their sole discretion. Upgrades refer to substantial changes to the Solution, the platform or infrastructure and are represented by major versioning numbers (example: move from 15.x to 15.x) UT – User Testing. Objective This is the Test Plan for the OregonBuysStatefirst system quarterly releases and yearly upgrades. The original project was the OregonBuysStatefirst Instance project. Establishment of the Instance as the 'backbone' for OregonBuysStatefirst allowed Secretary of State and Oregon Department of Forestry the opportunity to become the first agencies to implement OregonBuysStatefirst. They are able to utilize basic functionality such as Open Market Requisitions, Open Market Bids and Master Blanket Purchase orders, receive deliveries, approve invoices and make vendor payments through the R*STARS interface. Comment by Hornig, Tim: This is a nit but the PA differentiates “upgrades” from “maintenance releases” and “patches”. Consider defining “release” as a broader term covering all three…Comment by Hornig, Tim: Consider updating to reflect all agencies and a mixed bag of functionality (Phase 1 vs. Early Adopters vs. SOS/Forestry) and implementation of Marketplace.Comment by HARRIS DeAnna * DAS: I will update the wording This test plan should be followed for all quarterly releases for OregonBuysStatefirst. ScopeComment by Hornig, Tim: Consider adding more here – many of the notes under “schedule” below discuss scope. If scope is detailed here the “schedule” part can be more narrow. The scope of this test plan will change with every quarterly release or yearly upgrade, but the steps that will be taken for each release should remain the same PHI generally plans an Upgrade each year with a Release every quarter but may opt to do fewer Upgrades and Releases each year. No later than 20 business days prior to a planned version update to the UAT environment, PHI will develop and deliver a deployment plan to support the deployment of the update, testing, support, and training. The deployment plan will include the testing scope as well as the process by which defects will be managed. The Program Administrator will review the release notes and upon approval the OBE testing team will have 20 business days to test. If the OBE testing team identifies any defects, they will be emailed to the PHI UAT support email and PHI will have 10 business days to resolve the identified defects based on established specifications and service level agreements set forth in Exhibit F, Service Level Agreement and deploy as appropriate in a Patch to that Upgrade or Release. The OBE testing team will then have 10 business days to test and confirm the fixes. After a review of testing results and approval by Program Administrator via email to PHI, the Upgrade or Release will be deployed to the Production environment. If there is a Patch the same steps below will be followed. If critical Defects or vulnerabilities are discovered, PHI will notify the Program Administrator of the critical issue and provide a plan to remediate. PHI will provide details of the Patch prior to the installation of the fix into the UAT environment. The OBE testing team will have 10 business days to test the Patch. If defects are identified, PHI will have 5 business days to resolve the issues. Once issues are resolved, OBE testing team will have 5 business days to test and confirm the fixes. After a review of testing results and approval by the Program Administrator via email to PHI, the Patch will be deployed to the Production environment. Out of Scope The quarterly release or yearly upgrade release notes will sometimes contain items that are for another client or are for functionality that Oregon does not currently use. These items are out of scope and will not be tested by the OBE testing team ScheduleComment by Hornig, Tim: Looking at the content below consider changing the label to “process”. Maybe put a summary table at the end highlighting the schedule details after the process has been introduced? E.g., 1) PHI provides release notes, 2) Test Lead meets with testers, 3) testers test, etc.Comment by HARRIS DeAnna * DAS: I will update this section The schedule below will be the same for all Quarterly Upgrades and Yearly Releases)Comment by HARRIS DeAnna * DAS: You are correct, they are called hot fixes.Comment by Hornig, Tim: My question is whether “hot fixes” really follow the same schedule (20-10-10) – I would be surprised it they’re “hot”…Comment by HARRIS DeAnna * DAS: They do not follow the same schedule. Comment by Hornig, Tim: Is this always true? The PA talks about “hot fixes”(?) 1. Release Notes will be emailed to the Program Administrator from ZenDesk. A copy of the release notes will also be placed in PHI’s project Smartsheet – ePro 2021 20-10-10 Schedule (view only). 2. The OBE Testing Lead will schedule a meeting with all testers to review the notes prior to the UAT deployment a. Testers must identify which release note items they will test (OBE BA will track who is testing what item) b. Testers will identify what kind of test will be run for each item identified for testing. i. Procurement scenarios ii. AP Scenarios iii. VSR iv. Marketplace v. etc.Comment by Hornig, Tim: Are there new types – maybe Marketplace?Comment by HARRIS DeAnna * DAS: I will add marketplace, but without getting into specifics of the scenarios I think this is as detailed as I can get. 3. The R*STARS team will identify if the interface needs to be tested a. If yes, 3 successful batches are required (the R*STARS team can choose to accept less) b. Utility batches are set to run every Tuesday and Thursday evening – any additional batches will be will need to be approved by the R*STARS team and may have additional associated costs c. Prior to the batches running, the Program Administrator or IA will log in to the UAT environment and set the batch to run in OregonBuysStatefirst and clear any previous transactions ) 4. The day the upgrade/release is uploaded to UAT, PHI will provide an updated copy of the release notes, a demo of any new functionality and answer any testing questions. 5. During the first 20 business days, the OBE testing team will test all release items that were identified for Oregon. If there are any defects identified, PHI will have 10 business days to make changes or fixes 6. PHI will schedule a weekly meeting (Friday morning) with testers to discuss any issues or bugs a. Bugs that are found by testers will be emailed to the OBE Testing Lead with details and screenshots if necessary. b. OBE Testing Lead will email all bugs/issues to PHI help desk at
[email protected] c. PHI will review and either open internal ticket or respond to OBE Testing Lead with solution. 7. PHI will provide the Program Administrator with the updated release notes after their 10 days 8. The OBE Testing Lead will schedule a meeting with OBEtesting Team to review the notes with defect fixes Comment by HARRIS DeAnna * DAS: I will update thisComment by Hornig, Tim: Repeat from above – see note on “scope” 9. The R*STARS team will identify if the interface needs to be tested 10. The OBE testing team will have 10 business days to retest issues or bugs that were fixed along with any additional items that were built.Comment by Hornig, Tim: What about defects found in retesting any fixes? I understand they’re rare, but tests can fail…Comment by HARRIS DeAnna * DAS: I will update this wording. We do not get additional time if we find bugs in this last 10 days. If the