Methods for identifying emergent concepts in deep neural networks.

Räz, Tim (2023). Methods for identifying emergent concepts in deep neural networks. Patterns, 4(6), p. 100761. Cell Press 10.1016/j.patter.2023.100761

Preview

Text 1-s2.0-S266638992300106X-main.pdf - Published Version Available under License Creative Commons: Attribution-Noncommercial-No Derivative Works (CC-BY-NC-ND). Download (2MB) | Preview

The present perspective discusses methods to detect concepts in internal representations (hidden layers) of deep neural networks (DNNs), such as network dissection, feature visualization, and testing with concept activation vectors (TCAV). I argue that these methods provide evidence that DNNs are able to learn non-trivial relations between concepts. However, the methods also require users to specify or detect concepts via (sets of) instances. This underdetermines the meaning of concepts, making the methods unreliable. The problem could be overcome, to some extent, by systematically combining the methods and by using synthetic datasets. The perspective also discusses how conceptual spaces-sets of concepts in internal representations-are shaped by a trade-off between predictive accuracy and compression. I argue that conceptual spaces are useful, or even necessary, to understand how concepts are formed in DNNs but that there is a lack of method for studying conceptual spaces.

Interest & Impact

Downloads

19 since deposited on 06 Jul 2023
16 in the past 12 months

Citations

5 Citations in Web of Science ®
6 Citations in Scopus

Search

in Google Scholar™

Services

Actions (login required)

Edit item

Actions (login required)

Edit item