Frontiers of AI: Insights with Geoffrey Hinton, Toronto Tech Week 2025
Source: My personal notes from talk on theme of Relationship between Digital and Biological Intelligence on 2025-06-25
Introduction: Models
Section titled “Introduction: Models”Meaning of words and their relationships. Generative model training: Words - connect with feature vectors, help with predicting next words. Train model by correcting next words.
Over the years, models improved using neural nets, modelling natural language, feature vectors (embeddings), and transformers.
Lego analogy for how words work: Lego blocks can model any 3 dimensional shape approximately. Words are like Lego blocks which can be used to model anything in language. Words can combine like Lego blocks and “shake hands” with other words. Words combined can make any structure.
Super intelligence are more effective if they can set their own sub goals
Risks of Super Intelligence
Section titled “Risks of Super Intelligence”- Bad actors like governments can use them for people manipulation and wars (autonomous weapons, cyber attacks)
- Super intelligence can be deceptive similar to how humans behave under threats
Computation
Section titled “Computation”- Digital vs analog computation: current computers run same programs on different hardware with same output. Analog computation can give slightly different results.
- “Mortal computation” - when humans dies, all knowledge dies with them. Best solution is teacher teaching a student their knowledge. The student will adapt that knowledge
How efficient is weight or gradient sharing? and Computation
Section titled “How efficient is weight or gradient sharing? and Computation”Digital computation needs energy, but makes it easy for agents with the same model to share and learn by sharing weights and gradients
Human computation is energy efficient but difficult in sharing existing knowledge (weights, gradients).
Conclusion: if energy is available, digital computation is preferred
Can computers have subjective experience?
Section titled “Can computers have subjective experience?”They can when their perceptual sensors are changed/damaged/affected just like humans can have perceptions which are affected. For example, a chatbot with a vision sensor can have something in front of the sensor that affects its vision and when told will realize it had a subjective experience.
Panel Discussion (Geoffrey Hinton, Nike Frosst, Moderated by Nora Young)
Section titled “Panel Discussion (Geoffrey Hinton, Nike Frosst, Moderated by Nora Young)”Blockers to AI usage: data access, privacy
Large Language Models being transformative
Section titled “Large Language Models being transformative”- Question of whether they are more like humans in or less so
- Example practical applications improve productivity:
- Human computer interface can be driven using language and have the computer understand. LLM as an assistant and task executor
- Health care: AI analyzes health scans to help specialists
- Multi modal models - training with words, audio, images to generate content, similar to LLM
- Creativity of LLM and Humans: in the connections developed during training
How to guard against AI risks?
Public needs apply pressure to government to put in regulations on the technology and AI companies. Improve education and social systems around technology usage.