User:Ashughes: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
Line 42: Line 42:
* time saved vs manual testing?
* time saved vs manual testing?
* key deliverable such as a dashboard or audit?
* key deliverable such as a dashboard or audit?
===== Activity Stream =====
* '''Owners''': George Echim, Ioana Crisan, Paul Silaghi
* '''Code''': [https://github.com/mozilla/activity-stream github] (merged to [https://dxr.mozilla.org/mozilla-central/source/browser/extensions/activity-stream/test mozilla-central] weekly)
* '''Tracking''': [https://docs.google.com/spreadsheets/d/1aux1g-iUnsESFLd9tzmQp1ZskPTNbmt0uo1GdM7ZpRs/edit#gid=931456630 spreadsheet]
* '''Dashboard''': ?
* '''Status''':
** '''2018-04-03''': 31 Open , 13 Resolved, 29% Complete
<bugzilla>{"blocks":"1410333"}</bugzilla>


===== Bookmarks =====
===== Bookmarks =====

Revision as of 22:19, 6 April 2018

About

Ashughes-avatar.png

My name is Anthony Hughes (typically go by ashughes) and I work as a quality engineer on Mozilla's DOM team from Mozilla's office in Vancouver, Canada.

I was introduced to open source (and Mozilla) in 2006 by David Humphrey, a professor at Seneca College in Toronto, Ontario, Canada. After volunteering for a few years, I transitioned to being a paid contributor in 2010. Starting out as an add-on reviewer I became involved with QA (quality assurance), testing Firefox features, driving Firefox releases, developing strategies, and running test events for the community. I lead a QA strategy to ship the first version of Firefox Hello (WebRTC based calling in the browser) and developed experiments to test GPU Process (a foundational piece of the Quantum architecture).

I am currently a member of the Product Integrity group working on improvements in automation coverage and transitioning to a more robust tool for managing incoming requests.

When I'm not trying to make Firefox better, I'm usually hiking up some mountain or just traveling somewhere new, always with my camera in tow.

Contact Info

Contributions

2018

PI Requests v2

OKR
Drive creation of a proof-of-concept tool to replace the mailing list / spreadsheet (wiki)
Tasks
☑ Conduct user interviews
☑ Document requirements of the ideal system based on user feedback
☑ Fix low-hanging fruit within the current system
☑ Draft a workflow design encompassing the minimum viable end-to-end workflow
☑ Get feedback on workflow design from key stakeholders (Softvision, PI Management, Service Now)
☑ Revise MVP workflow design
⇒ Discuss implementation of MVP workflow design in Service Now -- Pending return of Sean Rich from PTO (2018-04-03)
☐ Review implementation of MVP workflow design in Service Now
☐ Test implementation of MVP workflow design in Service Now with real users

PI Automation

OKR: Lead project teams in developing and maintaining automated tests

What are the metrics for this project?

  • number of defects found by automation?
  • time spent on implementation of a specific test?
  • time spent on maintenance of a specific test?
  • time saved vs manual testing?
  • key deliverable such as a dashboard or audit?
Bookmarks
  • Owners: George Echim, Ioana Crisan, Paul Silaghi
  • Code: mozilla-central
  • Tracking: spreadsheet
  • Dashboard: ?
  • Status:
    • 2018-04-03: 6 Open, 14 Resolved, 30% Complete
Full Query
ID Summary Priority Status
1419390 Add automated test for "Searching for bookmarks from the Bookmarks Sidebar works properly" P3 RESOLVED
1423118 Add automated test for "Tags can be added to bookmarks using the Library" -- RESOLVED
1423137 Add automated test for "Bookmarks can be removed from the Library" P2 RESOLVED
1427701 Add automated test for "Tags can be added to bookmarks using the star-shaped button" P3 RESOLVED
1427755 Add automated test for "The name of a bookmark can be changed" P3 RESOLVED
1431382 Add automated test for "Tags can be added to bookmarks using the star-shaped button, the library and the sidebar" P1 RESOLVED
1431390 Add automated test for "Specific tags can be removed from a bookmark" P1 RESOLVED
1433900 Add automated test for "The location of a bookmark can be changed" P2 RESOLVED
1433901 Add automated test for "Bookmarks can be removed from the Bookmarks Toolbar and from the Library" P2 RESOLVED
1437433 Add automated test for "Websites can be bookmarked from a private window" P2 RESOLVED
1437476 Add automated test for "A bookmark can be loaded inside the Bookmarks Sidebar" P2 RESOLVED
1439832 Add automated test for "Bookmarks can be opened from the Bookmarks Menu" P2 RESOLVED
1439906 Add automated test for "Bookmark folders can be cut/copied via context menu" P2 RESOLVED
1439935 Add automated test for "Bookmarks can be backed up to a JSON file and restored back by importing JSON file" P2 RESOLVED
1440703 Add automated test to check that the Bookmarks Toolbar and Sidebar can be enabled from the Bookmarks Menu P2 RESOLVED
1441789 Add automated test to check that websites can be bookmarked via URL drag & drop onto the Bookmarks Toolbar and Sidebar P3 RESOLVED
1442157 Add automated test for " The removal of a bookmark can be undone" P3 RESOLVED
1442182 Add automated test for "Links can be bookmarked via context menu " P3 NEW
1442188 Add automated test for "The items from the Bookmarks Toolbar can be sorted by name" P3 NEW
1444912 Add automated test for "Bookmarks can be opened from the Library" -- RESOLVED

20 Total; 2 Open (10%); 18 Resolved (90%); 0 Verified (0%);


WebRTC
  • Owners: Roxana Robotin (reporting to Jan-Yvar)
  • Code: mozilla-central
  • Tracking: spreadsheet
  • Dashboard: ?
  • Status:
    • 2018-04-03: 6 Open, 0 Resolved, 0% Complete
Full Query
ID Summary Priority Status
1404994 Write unit tests for the *Conduits P3 NEW
1406372 Create mochitest to verify echo cancellation on getUserMedia P3 NEW
1406376 Create mochitest to verify noise suppression on getUserMedia P3 NEW
1406377 Create mochitest to verify auto gain control on getUserMedia P3 NEW
1406910 Add a mochitest for input-only audio capture P3 ASSIGNED
1412915 Create a loopback devices sink to source for automated testing on Windows P3 NEW

6 Total; 6 Open (100%); 0 Resolved (0%); 0 Verified (0%);


Firefox UI Tests
Owners: ?
Code: https://dxr.mozilla.org/mozilla-central/source/testing/firefox-ui/tests/
Dashboard: <link to results dashboard>
Documentation: Matt Howell explains the reason the Update tests are disabled. Functional and Puppeteer tests need to be audited.

Local Testrun - Linux x86_64 - 2018-03-21: 

SUMMARY
-------
 7:22.40 INFO passed: 86
 7:22.40 INFO failed: 1
 7:22.40 INFO todo: 3 (skipped: 1)
 7:22.40 INFO 
FAILED TESTS
-------
 7:22.40 INFO test_initial_download.py test_initial_download.TestSafeBrowsingInitialDownload.test_safe_browsing_initial_download
 7:22.40 SUITE_END
Next Steps: ☐ Set up a local environment to run the tests locally and see what happens
☐ Analyze the result of the tests
Benchmarking
Owners: Alin Rus (reporting to Milan)
Code: https://github.com/Top5Softvision/Top5_Benchmark
Dashboard: https://docs.google.com/spreadsheets/d/1LxEcm8Fz1tRU9s9HhIOlHdktO_a5UEEtw6bemfjq0h4/edit?usp=sharing
Documentation: https://docs.google.com/document/d/1UT8GTcVUQDYqCySNNhSoRBnIOiJa7TjN0GAkDyFLEFA/edit
Next Steps: ☐ Finish writing tests against Youtube
☐ Begin work on automated dashboard
Metrics-Graphics-GFX
Owners:
Code:
Dashboard:
Documentation:
Next Steps: ☐ Anthony to review project status with Milan

2017

2017 Q1

  • ☐ something to do
  • ☑ something done
  • ✓ something done
  • ✔ something done

☐ something to do
☑ something done
✓ something done
✔ something done

2016

2016 Q4

1. Prototype a dashboard of long-term Graphics stability trends
  • [DONE] architect a system to automate updating chart data periodically beyond the 6-month Socorro API limit
    • [DONE] create a python script to clone crash data from Socorro
    • [DONE] set up python script on a 24-hour cron job
  • [DONE] prototype a series of long-term charts to track:
    • [DONE] WebGL success rate - rpi3/#psd-webgl
    • [DONE] WebGL crash rate - rpi3/#psd-webgl-rate
    • [DONE] Shutdown crashes - rpi3/#psd-shutdown
    • [DONE] Startup crashes - rpi3/#psd-startup
    • [DONE] Driver crashes - rpi3/#psd-driver-rate
    • [CARRY OVER] High-volume crashes
    • [DONE] Overall Graphics crash rate - rpi3/#psd-gfx-rate
  • [DONE] prototype a dashboard to display all long-term graphs - See <server>/index.htm#psd-dashboard
  • [DONE] [add'l] Move infrastructure to a Mac Mini behind VPN
  • [CARRY OVER] host the dashboard on metrics.mozilla.com/gfx (if possible)
2. Prototype a small-scale, automated emunittest lab to improve discoverability of WebGL regressions in Nightly
  • [DONE] set up a machine to be used for testing on a daily basis
  • [DONE] get a script from Jukka Jylänki's to automate publishing test results to his local server in Finland
  • [DONE] test the script on a dummy rig at my desk in Vancouver
  • [DONE] let the machine run for several days to capture long-term results - See clbri.com
  • [CARRY OVER] optionally, get Jukka to create a mobile version of his test site that we can point to WebQA's Saucelabs instance
3. Utilize the Graphics test plan framework to validate shipping SKIA Content on Windows to Aurora in Firefox 52
  • document the test plan framework and telemetry experiment lessons learned
  • work with Mason to document milestones and release criteria
  • ensure these criteria are measured and met along the way
4. Maintenance tasks to ensure GFX quality does not slip

GPU Process

  • Develop a test plan to qualify MVP
  • Deploy a Telemetry Experiment in Nightly to A/B test stability (blocked by bug 1323304)

2016 Q3

Tasks
  • ...
Bugs


2016 Q2

Tasks
  • Prototyped a crash-stats dashboard with auto-updating charts
  • Conducted a talk at London all-hands (presentation)
  • Conducted Animometer benchmarking (Mac OS, Windows 7)
  • Created an add-on to integrate crash-stats visualization with Bugzilla (v0.1)
  • Executed Firefox 46 testing to support bug 1255281 (testplan)
  • Executed Firefox 48 testing to support bug 1260507 (testplan)
  • Set up weekly triage of Top Crashes, Cold Crashes, Cold Trackers, Untriaged, and Help Wanted bugs with a dashboard to track progress
Bugs


2016 Q1

  • Filed a bug to enable special character searches on Socorro (bug 1239402)
  • Prototyped a dashboard of Socorro data
  • Prototyped a dashboard of Bugzilla data
  • Executed a testrun to vet Developer Edition 45 for WinXP + D3D9 + E10S re bug 1237769
Bugs


2015

2015 Q4

  • Drafted documentation on graphics device/driver blocklisting [1]
  • Executed testruns to sanity check Firefox 43 and Firefox 44 in Aurora
  • Organized Android sanity testing for SKIA update (due Dec 7, 2015)
  • Proposed inclusion of GPU chipset information in Socorro, tracked via bug 1192351
  • Prototyped a graphics quality dashboard
  • Prototyped a periodic triage workflow to close old bugs and escalate lost issues
  • Prototyped a dashboard to measure engagement during testdays
Bugs


2015 Q3

  • Developed a process to create a mission statement of and for the community (result)
  • Conducted one-to-one meetings with peers and collect feedback to inform the mission statement
  • Drafted a recommendation for Hossain Al Ikram to participate in 2015 Global Gatherings
  • Provided advice to Jamie Charlton about planning Firefox OS testing events
  • Dogfooded Windows 10 to report blocker bugs ahead of release
  • Created a process for daily sanity checks via the lab in Toronto
  • Executed outsourced sanity checking for Firefox Aurora 41
  • Executed outsourced sanity checking for Firefox Aurora 42
  • Extended the sanity checking to volunteers via One & Done (results form)
  • Documented best practices on MDN (artifact)
  • Developed a catalog of graphics hardware to facilitate device-specific testing (artifact)
  • Created an internal hardware catalog as a proof-of-concept
  • Migrated the internal catalog to a
  • Improved readability of the inventory based on peer review
Bugs
Retrieved from "https://wiki.mozilla.org/index.php?title=User:Ashughes&oldid=1191937"