Meditations on First Deployment c c e e d d o o A Practical - PowerPoint PPT Presentation

@ @ A A x x S S a a u u Meditations on First Deployment c c e e d d o o A Practical Guide to Responsible Development EuroPython 2020 Alejandro Saucedo @AxSaucedo

@ my name is Alejandro Hello, A x S a u c e d o Engineering Director Seldon Technologies Chief Scientist The Institute for Ethical AI & ML Governing Council Member-at-Large Association for Computing Machinery Alejandro Saucedo @AxSaucedo

@ A x S The magic of programming a u c e d o You can wake up with an idea and have a prototype by the end of day/weekend.

@ A x S Software is eating the world a u c e d o The future wonders of the world will be running Python

@ @ A A Critical infrastructure increasingly x x S S a a u u c c e e d d depends on running software o o ...and regardless of the software / hardware abstractions, the impact will always be human, at an individual and societal level

@ Urgency vs Best Practice A x S a u AND c e d o Cybersecurity Attacks Software Outages Misuse of personal data Algorithmic Bias The impact of a bad solution can be worse than no solution at all

@ Responsibility Infrastructure A x S a u c e d o Department / Organisation ● High level Principles 3 ● Governing structure ● Aligned objectives Team / Delivery Process ● Escalation structure ● Cross functional skillset Key domain experts ● 2 ● Accountability structure ● Principled alignment ● Relevant delivery structure Individual Practitioner ● Technology best practices 1 ● Most relevant tools ● Competence in field Professional responsibility ●

@ Professional Responsibility A x S a u c e d o As software Empowered Unempowered developers we have a growing professional 😏 🤕 Ethical responsibility to our craft 😉 🤫 Unethical Ethical ~~ Ought to do good ● Empowered ~~ Know how to ●

@ Going beyond the algorithms A x S a u c e d o Large ethical challenges cannot fall on the Programming Domain shoulders of a single Expertise Expertise software developer Industry Standards Policy Expertise

@ End-to-end Approach A x S a u c e d o Principles & Guidelines High level guidelines that provide a 1 principled approach towards designing, building and operating machine learning. Industry standards & regulatory frameworks Practical guidelines that set the bar 2 for requirements around risk assessment and evaluation for Open Source Software machine learning systems Practical implementations of the best 3 practices on the infrastructure that provides the backbone to most applications.

@ Terminology A x S a u c e d o Ethics Principles Moral principles that govern a Fundamental truths or propositions person's behaviour or the that serve as the foundation for a conducting of an activity. system of belief or behaviour or for a chain of reasoning. Why not just follow existing rules? When dealing with new technologies/situations, there may just not be enough examples to base on, but practitioners will need to make decisions

@ Whose Ethics? A x S a u c e d o Eastern? Western? …? Philosophical Foundations ! = Current (Geo)political ecosystem Understanding underlying philosophical foundations allows us to understand where we come The individual, continuity, good, the from, to come to more powerful righteous, ... mutual agreements

@ @ Principles & Ethics Frameworks A A x x S S a a u u c c e e d d o o The ACM’s Code of Ethics The IEML’s Principles for & Professional Conduct Responsible AI

@ @ A A x x Strive to achieve high quality... S S a a u u c c Maintain high standards... e e d d Contribute to society and to human well-being... o o Know and respect existing rules... Avoid harm Accept and provide appropriate professional Be honest and trustworthy review Perform work only in areas of competence Be fair and take action not to discriminate Foster public awareness and understanding... Respect the work required to produce new ideas... Access computing and communication Respect privacy resources only when authorized Design and implement systems that are robustly Honor confidentiality and usably secure Principles = good for business and software!

@ Industry/Code Standards A x S a u c e d o Who sets code/industry standards? You! Who uses the industry standards? Maybe You! Standard: A repeatable, harmonised, agreed & documented way of doing and maybe them too... something

@ Standardisation Bodies A x S a u c You can get involved in the design and e d o development and use of standards

@ Open Source as Foundation A x S a u c e d o Open source is now becoming the backbone for critical infrastructure that runs our society Open Source Software Practical implementations of the best 3 practices on the infrastructure that provides the backbone to most applications.

@ Open Source as Policy A x S a u c e d o Principles & Guidelines High level guidelines that provide a 1 Principles are principled approach towards designing, building and operating machine learning. useless if the foundation is not in place to introduce and Open Source Software manage Practical implementations of the best 3 practices on the infrastructure that provides the backbone to most applications.

@ Open Source as Lead A x S a u c e d o Principles & Guidelines High level guidelines that provide a 1 Open source principled approach towards designing, building and operating machine learning. leaders are developing the core cogs that regulation Open Source Software depends on Practical implementations of the best 3 practices on the infrastructure that provides the backbone to most applications.

@ Open Source Foundations A x S a u c You can get involved on the design and e d o development and use of standards

@ Sidenote: Regulation A x S a u c e d o We all can agree: Bad regulation is BAD. However good regulation can be a catalyst for innovation through enforcement of best practices and mitigation of bad actors.

@ @ Software’s Massive Traction A A x x S S a a u u c c e e d d o o ● Internet Services ● Machine Learning Automation Cloud Native infrastructure ● Gaming and design tools ● ● Etc, etc, etc, etc Growth

@ @ Not all can be solved w code A A x x S S a a u u c c e e d d o o Problems in When you run the world around with a Relevant hammer solutions everything may Tech solutions look like a nail Software solutions

E.g The Challenge of our @ A x S a u Generation c e d o Societal Impact Economic Impact

@ And potentially not the last A x S a u c e d o https://medium.com/@amynoelle/flatten-the-climate-change-curve-2ed756eaa082

@ Ensuring the right solution A x S a u c e d o Before tackling a problem we should be able to identify how much of it is actually a software problem before actually writing code And whether the solution is even solving a problem

@ A x S a u c e d o Practical Deep Dive Production machine learning systems

@ @ Prod ML Systems are HARD A A x x S S a a u u c c e e d d o o Specialised Hardware (GPU, etc) Complex Dependency Graphs Compliance Reproducibility of components Last year’s talk on the challenges & landscape in ML: https://www.youtube.com/watch?v=Ynb6X0KZKxY

1 Human augmentation / review Principles for responsible AI 2 Bias evaluation capabilities 3 http://ethical.institute/principles.html Explainability by justification 4 Reproducible ops infrastructure 5 Displacement strategy 6 Practical statistical metrics 7 Trust by privacy 8 Security risks @ A x S a u c e d o

@ Procurement Framework A x S a u c e d o A set of templates for industry practitioners: Request for proposal ● ML maturity model ● Tender competition template ● http://ethical.institute/rfx.html

@ ML Maturity Model A x S a u c e d o From principles to a checklist Practical benchmarks Explainability by justification Each has a set of questions for Infrastructure for reproducible operations ● supplier compliance Data and model assessment processe Privacy enforcing infrastructure Top-bottom approach providing Operational process design ● red flags Change management capabilities Security risk processes http://ethical.institute/rfx.html