Monthly Archives: December 2016

Giving machine learning systems

Machine learning, which is the basis for most commercial artificial-intelligence systems, is intrinsically probabilistic. An object-recognition algorithm asked to classify a particular image, for instance, might conclude that it has a 60 percent chance of depicting a dog, but a 30 percent chance of depicting a cat.

At the Annual Conference on Neural Information Processing Systems in December, MIT researchers will present a new way of doing machine learning that enables semantically related concepts to reinforce each other. So, for instance, an object-recognition algorithm would learn to weigh the co-occurrence of the classifications “dog” and “Chihuahua” more heavily than it would the co-occurrence of “dog” and “cat.”

In experiments, the researchers found that a machine-learning algorithm that used their training strategy did a better job of predicting the tags that human users applied to images on the Flickr website than it did when it used a conventional training strategy.

“When you have a lot of possible categories, the conventional way of dealing with it is that, when you want to learn a model for each one of those categories, you use only data associated with that category,” says Chiyuan Zhang, an MIT graduate student in electrical engineering and computer science and one of the new paper’s lead authors. “It’s treating all other categories equally unfavorably. Because there are actually semantic similarities between those categories, we develop a way of making use of that semantic similarity to sort of borrow data from close categories to train the model.”

Zhang is joined on the paper by his thesis advisor, Tomaso Poggio, the Eugene McDermott Professor in the Brain Sciences and Human Behavior, and by his fellow first author Charlie Frogner, also a graduate student in Poggio’s group. Hossein Mobahi, a postdoc in the Computer Science and Artificial Intelligence Laboratory, and Mauricio Araya-Polo, a researcher with Shell Oil, round out the paper’s co-authors.

Close counts

To quantify the notion of semantic similarity, the researchers wrote an algorithm that combed through Flickr images identifying tags that tended to co-occur — for instance, “sunshine,” “water,” and “reflection.” The semantic similarity of two words was a function of how frequently they co-occurred.

Ordinarily, a machine-learning algorithm being trained to predict Flickr tags would try to identify visual features that consistently corresponded to particular tags. During training, it would be credited with every tag it got right but penalized for failed predictions.

The MIT researchers’ system essentially gives the algorithm partial credit for incorrect tags that are semantically related to the correct tags. Say, for instance, that a waterscape was tagged, among other things, “water,” “boat,” and “sunshine.” With conventional machine learning, a system that tagged that image “water,” “boat,” “summer” would get no more credit than one that tagged it “water,” “boat,” “rhinoceros.” With the researchers’ system, it would, and the credit would be a function of the likelihood that the tags “summer” and “sunshine” co-occur in the Flickr database.

Students decipher cryptography

Lincoln Laboratory recently unveiled LLCipher, a new outreach program for high school students. LLCipher is a one-week cryptography workshop that provides an introduction to modern cryptography — a math-based, theoretical approach to securing data.

Lessons in abstract algebra, number theory, and complexity theory provide students with the foundational knowledge needed to understand theoretical cryptography. Students build on that knowledge to construct provably secure encryption and digital signature schemes. On the last day, the students learn about some less mainstream developments in cryptography. Guest speakers cover various techniques that enable multiple entities to exchange data without disclosing to one another more data than necessary to perform a particular function. These include zero-knowledge proofs (proving a statement is true without revealing any information beyond the truth of the statement) and multiparty computation (computing a function over multiple parties’ inputs while keeping the inputs private).”

The course, held at Lincoln Laboratory’s Beaver Works facility near MIT campus, came about with the help of Bradley Orchard of the laboratory’s Advanced Sensor Systems and Test Beds Group and Sophia Yakoubov, Emily Shen, and David Wilson, all of the Secure Resilient Systems and Technology Group.

Orchard conceived the idea for LLCipher while teaching high-school students at the Russian School of Mathematics in Lexington, Massachusetts. He noticed a significant need for additional learning opportunities beyond what is offered in even the best high schools. Orchard noticed some very bright students were ready for material beyond calculus. “I thought it would be fun to offer a short introductory summer course for advanced high-school students,” he said. “I naturally thought of cryptography because it combines beautiful mathematics with powerful, useful, and fun techniques, and most importantly, aspects of cryptography are very accessible to advanced students.”

To design the course, Orchard asked for help from the laboratory’s management; John Wilkinson, leader of the Cyber Systems Assessments Group; and cryptography experts including Yakoubov, who, knowing how much she enjoyed teaching the CyberPatriot students, was eager to get involved.

“I cannot emphasize how difficult it is to design a short course that is effective, interesting, and fun for high-school students,” Orchard said. “Sophia did a superb job accomplishing this task, as evidenced by the enthusiasm and participation of the students. The students were engaged, asking questions, and demonstrating that they understood the material, and, most importantly, having fun.”

Chiamaka Agbasi-Porter of the Communications and Community Outreach Office handled the student applications and class logistics. Agbasi-Porter indicated that each of the 16 selected students were able to express their enjoyment of math through independent projects in their application to be a part of LLCipher. She felt the students for this pilot class were smart and enthusiastic. Yakoubov agreed: “The class was very interactive and the students were engaged in the topic; their questions and ideas made them a pleasure to teach.”

The first thing covered in class was encryption. The instructors started by explaining the one-time pad, which is a perfectly secure encryption scheme, meaning that an eavesdropping adversary cannot learn anything about the encrypted message no matter how much computing power he or she has. Perfectly secure encryption schemes have a significant drawback, however: The secret key shared by the message sender (typically called “Alice” in the industry) and message receiver (referred to as “Bob”) has to be at least as long as the message itself. Alice might want to send Bob very long messages, and might not know in advance how long these messages will be, so establishing a sufficiently long key might be problematic.