A few interesting books that explore this:
Edward T. Hall: The Silent Language and other books on Amazon
George Lakoff: Women, Fire, and Dangerous Things: What Categories Reveal about the Mind
George Lakoff: Philosophy In The Flesh: The Embodied Mind and Its Challenge to Western Thought
George Lakoff, Mark Johnson: Metaphors We Live By explores how metaphor shapes thought, perception, and philosophy.
Steve Andreas does a deep-dive into how scope and category shape perception, emotion, behavior, and therapy in this landmark two-volume set:
Six Blind Elephants: Understanding Ourselves and Each Other, Vol. 1: Fundamental Principles of Scope and Category
Six Blind Elephants: Understanding Ourselves and Each Other, Vol. 2: Applications and Explorations of Scope and Category