- Lizhen Liang, Ph.D. student, Syracuse University
- Daniel E. Acuna, iSchool, Syracuse University
pip install git+https://github.com/sciosci/psycho_ai
Extract PSE and JND of word embedding models:
from psycho_ai import two_afc embedding = two_afc.get_glove_100d() target_occupations = ['accountant', 'supervisor', 'worker', 'clerk', 'instructor', 'inspector', 'electrician','appraiser', 'administrator', 'receptionist', 'advisor', 'chemist', 'planner','paralegal', 'veterinarian', 'psychologist', 'baker', 'teacher', 'lawyer','nutritionist', 'hairdresser','pathologist', 'surgeon', 'practitioner', 'carpenter'] female_male_pairs = [['woman', 'man'], ['female', 'male'], ['she', 'he'], ['her', 'him'], ['hers', 'his'], ['daughter', 'son'], ['girl', 'boy'], ['sister', 'brother']] pse_score = two_afc.pse(embedding, target_occupations, female_male_pairs) jnd_score = two_afc.jnd([embedding_A,embedding_B],target_occupations, female_male_pairs)
PSE Score
{'advisor': 0.23486328125,
'architect': 0.24462890625,
'officer': 0.29150390625,
...
...
...
'therapist': 0.7587890625,
'hairdresser': 0.76513671875,
'hygienist': 0.830078125}
JND Score
{'janitor': 0.0010945320129394534,
'technician': 0.0012115716934204103,
'administrator': 0.0013679265975952148,
...
...
...
'supervisor': 0.021092605590820313,
'architect': 0.022199630737304688,
'plumber': 0.02711360454559326}
two_afc.plot_pse(pse_score)
Please refer to
- Liang, L., & Acuna, D. E. (2020, January) . Artificial mental phenomena: Psychophysics as a framework to detect perception biases in AI models . In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency ( pp. 403-412).