Timo Denk's Blog

Science, Programming, 3D-Printing, Music, Philosophy, and More…

Tag: Derivation

Linear Relationships in the Transformer’s Positional Encoding

In June 2017, Vaswani et al. published the paper “Attention Is All You Need” describing the “Transformer” architecture, which is a purely attention based sequence to sequence model. It can be applied to many tasks, such as language translation and text […]

January 22, 2019January 22, 2019by Timo Denk

Least Squares Derivation

The least squares optimization problem searches for a vector, that minimizes the euclidean norm in the following statement as much as possible: $$x_\text{opt}=\arg\min_x\frac{1}{2}\left\lVert Ax-y\right\rVert^2_2\,.$$This article explains how $x_\text{opt}=(A^\top A)^{-1}A^\top y$, the solution to the problem, can be derived and how […]

July 26, 2017December 9, 2017by Timo Denk

Find posts

About

Hi there! This blog is run by me, Timo. I am a Software Engineer living in Zürich, Switzerland. Please visit my portfolio website for more detailed information.

Editor’s choice

If you just came here and do not know where to go, I recommend looking into these posts:

  • Digit Span Test (most viewed)
  • Cubic Spline Interpolation (math-heavy)
  • Commute Time Tracking (nice read)
  • Linear Relationships in the Transformer’s Positional Encoding (machine learning)
  • Corsairs3D (pirate game – arrr!)

Archive

The archive page of this blog lists all posts in chronological order.

Email Newsletter

There is an email newsletter available. Once signed up, you will receive an email every time a new post is published. That is usually about once or twice a month.

Tags

3D 3D printing Arduino Art Blender Book C++ Computer Science Conference Convolutional network Deep learning DenkDevelopment Ender 3 Pro Functional programming Game Graph Hardware Haskell Historic History JavaScript Library Low-level Machine learning Math Microcontroller Miniature Modeling Music Neural network Node.js Painting Paper Philosophy Piano Poker Programming Raspberry Pi SAP Science Space Telegram Telegram bot Test Tools

Donate

I have decided to remove all ads from this page, because I just don't find it to be a nice way of making a few bucks a month.

If you enjoyed the content of my site, please consider donating a small amount of money through the link below.

Suggested values are $\sqrt{2}$, $\pi$, or the electron mass (without the $\times10^{-31}\text{kg}$), but you can also choose an arbitrary amount.

Navigation

Home · Contact · Legal
Proudly powered by WordPress | Theme: Sin by vsFish.

Cookies

This site uses cookies. More