University of Pittsburgh

News Archive

April 21 2021
ISP Director Vanathi Gopalakrishnan referenced in AI Magazine

Congratulations to ISP director, Vanathi Gopalakrishnan, on her contribution to an article in AI Magazine, wherein she has included an original definition for "Intelligence" and its impact for "General AI."

See CMU CS press release here:

March 04 2021
Congratulations to Profs. Sera Linardi and Yu-Ru Lin for their Pitt Momentum Seed Grant

Congratulations to Profs. Sera Linardi and Yu-Ru Lin for their Pitt Momentum Seed Grant, entitled: Community-centric Building of Digital Infrastructures Against Systemic Oppression: G2A police misconduct complaint support system!

February 28 2021
Dr. Adriana Kovashka receives NSF CAREER award

Congratulations to Dr. Adriana Kovashka for her recent NSF CAREER award, for Natural Narratives and Multimodal Context as Weak Supervision for Learning Object Categories

This project develops a framework to train computer vision models for detection of objects from weak, naturally-occurring supervision of language (text or speech) and additional multimodal signals. It considers dynamic settings, where humans interact with their visual environment and refer to the encountered objects, e.g., ?Carefully put the tomato plants in the ground? and ?Please put the phone down and come set the table,? and captions written for a human audience to complement an image, e.g., news article captions. The challenge of using such language-based supervision for training detection systems is that along with useful signal, the speech contains many irrelevant tokens. The project will benefit society by exploring novel avenues for overcoming this challenge and reducing the need for expensive and potentially unnatural crowdsourced labels for training. It has the potential to make object detection systems more scalable and thus more usable by a broad user base in a variety of settings. The resources and tools developed would allow natural, lightweight learning in different environments, e.g., different languages or types of imagery where the well-known object categories are not useful or where there is a shift in both the pixels as well as the way in which humans refer to objects (different cultures, medicine, art). This project opens possibilities for learning in vivo rather than in vitro; while the focus here is on object categories, multimodal weak supervision is useful for a larger variety of tasks. Research and education are integrated through local community outreach and research mentoring for students from lesser-known universities, new programs for student training including honing graduate students' writing skills, and development of interactive educational modules and demos based on research findings. 

This project creatively connects two domains, vision-and-language, and object detection, and pioneers training of object detection models with weak language supervision and a large vocabulary of potential classes. The impact of noise in the language channel will be mitigated through three complementary techniques that model visual concreteness of words, to what extent the text refers to the visual environment it appears with, and whether the weakly-supervised models that are learned are logically consistent. Two complementary word-region association mechanisms will be used (metric learning and cross-modal transformers), whose application is novel for weakly-supervised detection. Importantly, to make detection feasible, not only the semantics of image-text pairs, but their discourse relationship, will be captured. To facilitate and disambiguate the association of words to a physical environment, the latter will be represented through additional modalities, namely sound, motion, depth and touch, which are either present in the data or estimated. This project advances knowledge of how multimodal cues contextualize the relation between image and text; no prior work has modeled image-text relationships along multiple channels (sound, depth, touch, motion). Finally, to connect the appearance of objects to the purpose and use of these objects, relationships between objects, properties and actions will be semantically organized in a graph, and grammars to represent activities involving objects will be extracted, still maintaining the weakly-supervised setting.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.


December 22 2020
Congratulations to Prof. Sera Linardi for her recent talk in the Harvard Center for Research and Computation for Society (CRCS)

Our ISP faculty, Prof. Sera Linardi, has been recognized by the CRCS!Congratulations to Prof. Sera Linardi for her recent talk in the Harvard Center for Research and Computation for Society (CRCS) AI for Social Good seminar series entitled:"Lessons from Imperfect Attempts to Serve (from Academia)"

April 13 2020
Provost Predoctoral Fellowships

Two ISP students won the Provost Predoctoral Fellowships for 2020-2021:

The Provost Predoctoral Fellowships are offered by the University of Pittsburgh to students in many graduate programs. Typically, ISP students are nominated to apply for the Provost Predoctoral Fellowships after their second year of graduate study. 

March 04 2020
Congratulations to Profs. Diane Litman and Maria Chikina for winning the Provost Faculty Awards.

Our ISP faculties, Drs. Litman and Chikina have received two of the 2020 Provost Faculty Awards. Congratulations to Dr. Litman on her Award for Excellence in Doctoral Mentoring and Dr. Chikina for her Chancellor's Distinguished Research Award. (

January 29 2020
Congratulations to ISP Alumnus Prof. Vincent Aleven on receiving LRDC's distinguished alumni award.

Vincent Aleven received the LRDC's 2019 Distinguished Alumni Award, and presented his research on AI used in classrooms. Here is the link for his full speech:

December 19 2019
Congratulations to Dr. Kevin Ashley and Jaromir Savelka for winning the Jurix 2019 Best Student Paper Awards.

Jurix 2019 was the 32nd International Conference on Legal Knowledge and Information Systems which was held in Spain this December. Congratulation to our ISP faculty, Dr. Kevin Ashley, and ISP student, Jaromir Savelka, for winning the Best Student Award in this conference for their paper 'Computer-Assisted Creation of Boolean Search Rules for Text Classification in the Legal Domain'. 

December 17 2019
What Will the 2020s Bring for AI?

Check this interesting interview with two of our ISP faculties Dr. Ervin Sejdic and Dr. Diane Litman on Pittwire:

September 14 2019
Congratulations ISP Alumnus Prof. Bruce McLaren for getting Fulbright Scholarship to further educational technology in Chile.

The goal of the Fulbright project is to help further the development of educational technology in Chile, both as a research discipline and as a practical instructional tool for schools. Prof. McLaren Scholarship was awarded for him to teach a course on Educational Technology at the Universidad Téchnica Federico Santa Maria in Valparaiso, Chile.

Copyright 2009–2021 | Send feedback about this site