S T A T E   O F   N E W   Y O R K
 ________________________________________________________________________
 
                                   9430
 
                           I N  A S S E M B L Y
 
                              March 14, 2024
                                ___________
 
 Introduced  by  M. of A. OTIS -- read once and referred to the Committee
   on Science and Technology
 
 AN ACT to amend the state technology law, in relation to automated deci-
   sion-making by state agencies
 
   THE PEOPLE OF THE STATE OF NEW YORK, REPRESENTED IN SENATE AND  ASSEM-
 BLY, DO ENACT AS FOLLOWS:
 
   Section  1.  Short title.  This act shall be known and may be cited as
 the "legislative oversight of automated  decision-making  in  government
 act (LOADinG Act)".
   §  2. The state technology law is amended by adding a new article 4 to
 read as follows:
                                 ARTICLE IV
               AUTOMATED DECISION-MAKING IN STATE GOVERNMENT
 SECTION 401. DEFINITIONS.
         402. USE OF AUTOMATED DECISION-MAKING SYSTEMS BY AGENCIES.
         403. IMPACT ASSESSMENTS.
   § 401. DEFINITIONS. FOR THE PURPOSE OF THIS ARTICLE:
   1. "AUTOMATED DECISION-MAKING SYSTEM" SHALL  MEAN  ANY  SOFTWARE  THAT
 USES  ALGORITHMS, COMPUTATIONAL MODELS, OR ARTIFICIAL INTELLIGENCE TECH-
 NIQUES, OR A COMBINATION THEREOF, TO AUTOMATE, SUPPORT, OR REPLACE HUMAN
 DECISION-MAKING AND SHALL  INCLUDE,  WITHOUT  LIMITATION,  SYSTEMS  THAT
 PROCESS  DATA, AND APPLY PREDEFINED RULES OR MACHINE LEARNING ALGORITHMS
 TO  ANALYZE  SUCH  DATA,  AND  GENERATE  CONCLUSIONS,   RECOMMENDATIONS,
 OUTCOMES,  ASSUMPTIONS,  PROJECTIONS,  OR PREDICTIONS WITHOUT MEANINGFUL
 HUMAN REVIEW AND DISCRETION. "AUTOMATED  DECISION-MAKING  SYSTEM"  SHALL
 NOT  INCLUDE  ANY  SOFTWARE  USED PRIMARILY FOR BASIC COMPUTERIZED PROC-
 ESSES, SUCH AS CALCULATORS,  SPELLCHECK  TOOLS,  AUTOCORRECT  FUNCTIONS,
 SPREADSHEETS,  ELECTRONIC  COMMUNICATIONS, OR ANY TOOL THAT RELATES ONLY
 TO INTERNAL MANAGEMENT AFFAIRS SUCH AS ORDERING OFFICE SUPPLIES OR PROC-
 ESSING PAYMENTS, AND THAT DO NOT MATERIALLY AFFECT  THE  RIGHTS,  LIBER-
 TIES, BENEFITS, SAFETY OR WELFARE OF ANY INDIVIDUAL WITHIN THE STATE.
   2.  "STATE AGENCY" SHALL MEAN ANY DEPARTMENT, PUBLIC AUTHORITY, BOARD,
 BUREAU, COMMISSION, DIVISION, OFFICE, COUNCIL, COMMITTEE OR  OFFICER  OF
 THE STATE. SUCH TERMS SHALL NOT INCLUDE THE LEGISLATURE OR JUDICIARY.
 
  EXPLANATION--Matter in ITALICS (underscored) is new; matter in brackets
                       [ ] is old law to be omitted.
              

                                                            LBD11734-04-4
 A. 9430                             2
 
   3. "PUBLIC ASSISTANCE BENEFIT" SHALL MEAN ANY SERVICE OR PROGRAM WITH-
 IN  THE  CONTROL OF THE STATE, OR BENEFIT PROVIDED BY THE STATE TO INDI-
 VIDUALS OR HOUSEHOLDS, INCLUDING BUT NOT LIMITED TO  PUBLIC  ASSISTANCE,
 CASH  ASSISTANCE,  GRANTS,  CHILD  CARE  ASSISTANCE, HOUSING ASSISTANCE,
 UNEMPLOYMENT  BENEFITS,  TRANSPORTATION  BENEFITS, EDUCATION ASSISTANCE,
 DOMESTIC VIOLENCE SERVICES, AND ANY OTHER ASSISTANCE OR  BENEFIT  WITHIN
 THE  AUTHORITY  OF  THE  STATE TO GRANT TO INDIVIDUALS WITHIN THE STATE.
 THIS SHALL NOT INCLUDE ANY FEDERAL PROGRAM THAT IS ADMINISTERED  BY  THE
 FEDERAL GOVERNMENT OR THE STATE.
   §  402.  USE  OF AUTOMATED DECISION-MAKING SYSTEMS BY AGENCIES. 1. ANY
 STATE AGENCY, OR ANY ENTITY ACTING ON BEHALF OF SUCH  AGENCY,  SHALL  BE
 PROHIBITED FROM, DIRECTLY OR INDIRECTLY, UTILIZING OR APPLYING ANY AUTO-
 MATED  DECISION-MAKING  SYSTEM  IN  PERFORMING ANY FUNCTION THAT: (A) IS
 RELATED TO THE DELIVERY OF ANY PUBLIC ASSISTANCE BENEFIT; (B) WILL  HAVE
 A  MATERIAL  IMPACT ON THE RIGHTS, CIVIL LIBERTIES, SAFETY OR WELFARE OF
 ANY INDIVIDUAL WITHIN THE STATE;  OR  (C)  AFFECTS  ANY  STATUTORILY  OR
 CONSTITUTIONALLY  PROVIDED  RIGHT OF AN INDIVIDUAL; UNLESS SUCH UTILIZA-
 TION OR APPLICATION OF THE AUTOMATED DECISION-MAKING SYSTEM  IS  SPECIF-
 ICALLY AUTHORIZED IN LAW.
   2. NO STATE AGENCY SHALL AUTHORIZE ANY PROCUREMENT, PURCHASE OR ACQUI-
 SITION  OF  ANY  SERVICE  OR  SYSTEM UTILIZING, OR RELYING ON, AUTOMATED
 DECISION-MAKING SYSTEMS PROHIBITED IN SUBDIVISION ONE OF  THIS  SECTION,
 EXCEPT WHERE THE USE OF SUCH SYSTEM IS SPECIFICALLY AUTHORIZED IN LAW.
   §  403.  IMPACT ASSESSMENTS. 1. NO STATE AGENCY SHALL UTILIZE OR APPLY
 ANY AUTOMATED DECISION-MAKING SYSTEM UNLESS  THE  STATE  AGENCY,  OR  AN
 ENTITY  ACTING  ON  BEHALF OF SUCH STATE AGENCY, SHALL HAVE CONDUCTED AN
 IMPACT ASSESSMENT FOR THE APPLICATION AND USE OF  SUCH  AUTOMATED  DECI-
 SION-MAKING  SYSTEM.  FOLLOWING  THE  FIRST IMPACT ASSESSMENT, AN IMPACT
 ASSESSMENT SHALL BE CONDUCTED AT LEAST ONCE EVERY TWO YEARS.  AN  IMPACT
 ASSESSMENT  SHALL BE CONDUCTED PRIOR TO ANY MATERIAL CHANGE TO THE AUTO-
 MATED DECISION-MAKING SYSTEM THAT MAY CHANGE THE OUTCOME  OR  EFFECT  OF
 SUCH SYSTEM.  SUCH IMPACT ASSESSMENTS SHALL INCLUDE:
   (A)  A  DESCRIPTION OF THE OBJECTIVES OF THE AUTOMATED DECISION-MAKING
 SYSTEM;
   (B) AN EVALUATION OF THE  ABILITY  OF  THE  AUTOMATED  DECISION-MAKING
 SYSTEM TO ACHIEVE ITS STATED OBJECTIVES;
   (C)  A DESCRIPTION AND EVALUATION OF THE OBJECTIVES AND DEVELOPMENT OF
 THE AUTOMATED DECISION-MAKING INCLUDING:
   (I) A SUMMARY OF THE UNDERLYING ALGORITHMS, COMPUTATIONAL  MODES,  AND
 ARTIFICIAL  INTELLIGENCE  TOOLS THAT ARE USED WITHIN THE AUTOMATED DECI-
 SION-MAKING SYSTEM; AND
   (II) THE DESIGN AND TRAINING DATA USED TO DEVELOP THE AUTOMATED  DECI-
 SION-MAKING SYSTEM PROCESS;
   (D) TESTING FOR:
   (I)  ACCURACY, FAIRNESS, BIAS AND DISCRIMINATION, AND AN ASSESSMENT OF
 WHETHER THE USE OF THE AUTOMATED DECISION-MAKING SYSTEM PRODUCES DISCRI-
 MINATORY RESULTS ON THE BASIS OF A CONSUMER'S OR A CLASS  OF  CONSUMERS'
 ACTUAL  OR  PERCEIVED RACE, COLOR, ETHNICITY, RELIGION, NATIONAL ORIGIN,
 SEX, GENDER, GENDER IDENTITY, SEXUAL ORIENTATION, FAMILIAL STATUS, BIOM-
 ETRIC INFORMATION, LAWFUL SOURCE OF INCOME, OR DISABILITY  AND  OUTLINES
 MITIGATIONS  FOR  ANY  IDENTIFIED  PERFORMANCE  DIFFERENCES  IN OUTCOMES
 ACROSS RELEVANT GROUPS IMPACTED BY SUCH USE;
   (II) ANY CYBERSECURITY VULNERABILITIES  AND  PRIVACY  RISKS  RESULTING
 FROM THE DEPLOYMENT AND USE OF THE AUTOMATED DECISION-MAKING SYSTEM, AND
 THE DEVELOPMENT OR EXISTENCE OF SAFEGUARDS TO MITIGATE THE RISKS;
 A. 9430                             3
 
   (III)  ANY PUBLIC HEALTH OR SAFETY RISKS RESULTING FROM THE DEPLOYMENT
 AND USE OF THE AUTOMATED DECISION-MAKING SYSTEM;
   (IV)  ANY REASONABLY FORESEEABLE MISUSE OF THE AUTOMATED DECISION-MAK-
 ING SYSTEM AND THE DEVELOPMENT OR EXISTENCE OF SAFEGUARDS  AGAINST  SUCH
 MISUSE;
   (E)  THE EXTENT TO WHICH THE DEPLOYMENT AND USE OF THE AUTOMATED DECI-
 SION-MAKING SYSTEM REQUIRES INPUT OF SENSITIVE AND  PERSONAL  DATA,  HOW
 THAT  DATA IS USED AND STORED, AND ANY CONTROL USERS MAY HAVE OVER THEIR
 DATA; AND
   (F) THE NOTIFICATION MECHANISM OR PROCEDURE, IF ANY, BY WHICH INDIVID-
 UALS IMPACTED BY THE UTILIZATION OF THE AUTOMATED DECISION-MAKING SYSTEM
 MAY BE NOTIFIED OF THE USE OF SUCH AUTOMATED DECISION-MAKING SYSTEM  AND
 OF  THE  INDIVIDUAL'S  PERSONAL  DATA,  AND INFORMED OF THEIR RIGHTS AND
 OPTIONS RELATING TO SUCH USE.
   2. NOTWITHSTANDING THE PROVISIONS OF THIS ARTICLE OR ANY OTHER LAW, IF
 AN IMPACT ASSESSMENT FINDS THAT  THE  AUTOMATED  DECISION-MAKING  SYSTEM
 PRODUCES DISCRIMINATORY OR BIASED OUTCOMES, THE STATE AGENCY SHALL CEASE
 ANY  UTILIZATION,  APPLICATION,  OR FUNCTION OF SUCH AUTOMATED DECISION-
 MAKING SYSTEM, AND OF ANY INFORMATION PRODUCED USING SUCH SYSTEM.
   3. ANY IMPACT ASSESSMENT CONDUCTED PURSUANT TO THIS SUBDIVISION  SHALL
 BE SUBMITTED TO THE GOVERNOR, THE TEMPORARY PRESIDENT OF THE SENATE, AND
 THE  SPEAKER OF THE ASSEMBLY AT LEAST THIRTY DAYS PRIOR TO THE IMPLEMEN-
 TATION OF THE AUTOMATED DECISION-MAKING SYSTEM THAT IS  THE  SUBJECT  OF
 SUCH  ASSESSMENT.  THE  IMPACT STATEMENT OF AN AUTOMATED DECISION-MAKING
 SYSTEM THAT IS APPROVED AND UTILIZED, SHALL BE PUBLISHED ON THE  WEBSITE
 OF  THE  RELEVANT AGENCY. IF THE STATE AGENCY MAKES A DETERMINATION THAT
 THE DISCLOSURE OF ANY INFORMATION  REQUIRED  IN  THE  IMPACT  ASSESSMENT
 WOULD RESULT IN A SUBSTANTIAL NEGATIVE IMPACT ON HEALTH OR SAFETY OF THE
 PUBLIC, INFRINGE UPON THE PRIVACY RIGHTS OF INDIVIDUALS, OR SIGNIFICANT-
 LY IMPAIR THE STATE AGENCY'S ABILITY TO PROTECT ITS INFORMATION TECHNOL-
 OGY OR OPERATIONAL ASSETS, IT MAY REDACT SUCH INFORMATION, PROVIDED THAT
 AN  EXPLANATORY  STATEMENT ON THE PROCESS BY WHICH THE STATE AGENCY MADE
 SUCH DETERMINATION IS PUBLISHED ALONG WITH THE REDACTED  IMPACT  ASSESS-
 MENT.
   §  3.  Disclosure  of  existing automated decision-making systems. Any
 state agency, that directly or indirectly, utilizes an  automated  deci-
 sion-making  system,  as  defined in section 401 of the state technology
 law, shall submit to the legislature a disclosure on  the  use  of  such
 system, no later than one year after the effective date of this section.
 Such disclosure shall include:
   (a)  a description of the automated decision-making system utilized by
 such agency;
   (b) a list of any software vendors related  to  such  automated  deci-
 sion-making system;
   (c) the date that the use of such system began;
   (d)  a  summary  of  the  purpose  and use of such system, including a
 description  of  human  decision-making  and  discretion  supported   or
 replaced by the automated decision-making system;
   (e)  whether  any impact assessments for the automated decision-making
 system were conducted and the dates and summaries of the results of such
 assessments where applicable; and
   (f) any other information deemed relevant by the agency.
   § 4. This act shall take effect immediately, provided that section two
 of this act shall take effect one year after it shall have become a law.