#  EVP-AIIP program pilots conclude with valuable learnings on the application of AI 

 





March 03, 2025

 

 

The [Executive Vice President Artificial Intelligence Innovation Program](https://www.huit.harvard.edu/aiip) (EVP-AIIP) is intended to encourage the exploratory use of generative AI to improve operational efficiency and administrative systems and processes. During its first round, the program enabled the completion of eight funded projects. The EVP-AIIP provided Central Administration and University stakeholders with valuable insight on opportunities, limitations, and examples where AI ended up not being the answer for an improvement strategy.

The following pilots demonstrated significant potential for improving efficiency, automation, and data-driven decision-making across various administrative and operational domains; however, common challenges included AI tool limitations, the need for structured data (input), human oversight, and technical expertise gaps.

1. ### **Service RightNow (AI-Enabled Customer Service Ticketing)**
    
    **Unit:** Harvard University IT (HUIT)  
    **Team:** Joyce Kaplan, Maria Van Den Bosch, Andrea Sexton  
    **Tool(s):** IBM Watson
    
    Summary: This project explored AI-driven automation to respond to ServiceNow customer service tickets, aiming for productivity gains and administrative efficiencies. IBM Watson was integrated, with early success in handling 20% of supplier onboarding inquiries; however, implementation challenges included difficulty in learning and retaining AI tool skills. The project highlighted the need for a dedicated AI expert to support operational teams, demonstrating that while AI can enhance efficiency, proper expertise and strategic planning are crucial for long-term adoption.
2. ### **AI for Reunion Customer Service**
    
    **Unit:** Harvard Alumni Affairs & Development  
    **Team:** Nancy Conroy  
    **Tool(s):** Robotic Process Automation / UI Path
    
    Summary: The project aimed to enhance reunion event customer service using AI for email classification and response automation. While it successfully identified key information, service categories, and escalation points, challenges included using incorrect datasets and integration issues with co-managed inboxes. The project underscored the need for improved triage processes and AI applications in general inquiry handling, showcasing AI’s potential for large-scale customer service but emphasizing data accuracy and operational bandwidth concerns.
3. ### **HUHS FirstAIde (AI-Enabled Healthcare Chatbot)**
    
    **Unit:** Harvard University Health Services (HUHS)  
    **Team:** Jason Ward, David Naimark, Richard Wells, Mitch Hamilton  
    **Tool(s):** Microsoft CoPilot
    
    Summary: This pilot sought to develop a private chatbot for HUHS Member Services to answer health plan questions. The project encountered significant limitations across chatbot platforms, struggling with complex queries and AI hallucinations. While it improved knowledge base content for Member Services, the chatbot was not sufficiently reliable. A major takeaway was the necessity of HIPAA-compliant solutions for production systems, highlighting both the promise and difficulty of implementing AI in healthcare settings.
4. ### **AI-Assisted Contract Drafting**
    
    **Unit:** Office of Contract Services  
    **Team:** Elizabeth Copeland, Max Hurwitz, Kari Krengel  
    **Tool(s):** Many tested, none selected
    
    Summary: This pilot planned to use AI for contract drafting but faced challenges with generative AI's inconsistency. The project pivoted to contract risk analysis and redlining, but the vast diversity of contracts (75% on third-party paper) limited AI learning. Despite these hurdles, the project broadened AI awareness in contract management and emphasized the need for more mature AI solutions in legal drafting. It reinforced that AI in contract work is still evolving and requires structured data to be most effective.
5. ### **AI for Data Use Agreements (DUAs)**
    
    **Unit:** Office of the Vice Provost for Research  
    **Team:** Ara Tahmassian, Chris Stubbs  
    **Tool(s):** ChatGPT / Open AI Assistant
    
    Summary: The project used AI to analyze and flag problematic contract clauses in Data Use Agreements, aiming for efficiency in legal review. Despite limitations in AI training data, collaboration with the AI Community of Practice (AI CoP) led to a functional tool that provided useful summaries and negotiation effort estimates while AI-powered redlining remains a future goal. The project showcased AI’s value in legal workflows while highlighting the need for structured knowledge bases and improved AI reliability.
6. ### **AI for Administrative Learning Resources**
    
    **Unit:** Research Administration  
    **Team:** Christyne Anderson, Tracey Westervelt, Kyli White  
    **Tool(s):** ChatGPT, Microsoft CoPilot, Vyond, ElevenLabs, ArticulateAI
    
    Summary: This initiative used AI to expedite training content creation for research administration, reducing subject matter expert (SME) workload from 40 hours per course. AI-assisted tools streamlined content generation but still required 2-3 hours of human review. The project demonstrated AI’s strong potential for content development but affirmed the necessity of human oversight for quality assurance. AI-driven training tools could be widely adopted if licensing is made more accessible (cost and audience).
7. ### **AI-TIES (AI-Driven Targeted Information Extraction & Summarization)**
    
    **Unit:** Harvard Planning & Project Management  
    **Team:** Giovanni Zambotti, Jim Nelson, Megan Mandosa-Hayes  
    **Tool(s):** ChatGPT / Open AI Assistant
    
    Summary: This project automated the extraction of regulatory updates from municipal websites for Harvard-related content. Initial AI tools failed to meet web scraping needs, but a custom Python solution enabled PDF-based extraction. AI tools successfully structured a knowledge base for regulatory information. The project demonstrated AI’s utility in structured data extraction but emphasized challenges in AI reliability and hallucinations. The future focus includes refining metadata structuring and Retrieval-Augmented Generation (RAG).
8. ### **AI for HR Policies**
    
    **Unit:** Harvard Human Resources (HHR)  
    **Team:** Jack Wilcox  
    **Tool(s):** Microsoft CoPilot
    
    Summary: This initiative leveraged AI to analyze and improve HR policy accessibility, generate FAQs, and enhance search functions. It succeeded in correcting policy content and improving accessibility but faced technical barriers. The project highlighted AI’s potential in HR knowledge management but underscored the need for better technical expertise and policy integration across the university. Plans include expanding AI’s role in state employment data analysis.