A few interesting books that explore this:

Edward T. Hall: The Silent Language and other books on Amazon

George Lakoff: Women, Fire, and Dangerous Things: What Categories Reveal about the Mind

George Lakoff: Philosophy In The Flesh: The Embodied Mind and Its Challenge to Western Thought

George Lakoff, Mark Johnson: Metaphors We Live By explores how metaphor shapes thought, perception, and philosophy.

Steve Andreas does a deep-dive into how scope and category shape perception, emotion, behavior, and therapy in this landmark two-volume set:

Six Blind Elephants: Understanding Ourselves and Each Other, Vol. 1: Fundamental Principles of Scope and Category

Six Blind Elephants: Understanding Ourselves and Each Other, Vol. 2: Applications and Explorations of Scope and Category