Higham and Higham's Deep Learning: An Introduction for Applied
Mathematicians is a fairly short introduction to neural networks that is written with mathematicians in mind.
Another reference I would recommend is Shalev-Shwartz and Ben-David's Understanding Machine Learning: From Theory to Algorithms, particularly chapter 20.
Both of the above approach ...
The online textbook Neural Networks and Deep Learning by Michael Nielsen is quite good for mathematical content. It assumes the reader understands calculus, but requires no prior knowledge of machine learning. The book's own introduction explains fairly well what the book is about:
The purpose of this book is to help you master the core concepts of neural ...
@kaya3's comment links an excellent e-book, which describes the math in fair detail.
I'll never get tired of recommending 3blue1brown's amazing video series that both motivates the problem and gives basic intuition but doesn't shortchange the math: https://www.3blue1brown.com/neural-networks