Excerpt from this article:
Joy Buolamwini of the Massachusetts Institute of Technology will present work which suggests it is true.
Ms Buolamwini and her colleague Timnit Gebru looked at three sex-recognition systems, those of IBM, Microsoft and Face++. They tested these on a set of 1,270 photographs of parliamentarians from around the world and found that all three classified lighter faces more accurately than darker ones. All also classified males more accurately than females. IBM’s algorithm, for example, got light male faces wrong just 0.3% of the time. That compared with 34.7% of the time for dark female faces. The other two systems had similar gulfs in their performances. Probably, this bias arises from the sets of data the firms concerned used to train their software. Ms Buolamwini and Ms Gebru could not, however, test this because those data sets are closely guarded.
IBM has responded quickly. It said it had retrained its system on a new data set for the past year, and that this had greatly improved its accuracy.