Timo Denk's Blog

Science, Programming, 3D-Printing, Music, Philosophy, and More…

Tag: Derivation

Linear Relationships in the Transformer’s Positional Encoding

In June 2017, Vaswani et al. published the paper “Attention Is All You Need” describing the “Transformer” architecture, which is a purely attention based sequence to sequence model. It can be applied to many tasks, such as language translation and text […]

January 22, 2019January 22, 2019by Timo Denk

Least Squares Derivation

The least squares optimization problem searches for a vector, that minimizes the euclidean norm in the following statement as much as possible: $$x_\text{opt}=\arg\min_x\frac{1}{2}\left\lVert Ax-y\right\rVert^2_2\,.$$This article explains how $x_\text{opt}=(A^\top A)^{-1}A^\top y$, the solution to the problem, can be derived and how […]

July 26, 2017December 9, 2017by Timo Denk

Find posts

About

Hi there! This blog is run by me, Timo. I am a Software Engineer living in Zürich, Switzerland. Please visit my portfolio website for more detailed information.

Editor’s choice

If you just came here and do not know where to go, I recommend looking into these posts:

  • Digit Span Test (most viewed)
  • Cubic Spline Interpolation (math-heavy)
  • Commute Time Tracking (nice read)
  • Linear Relationships in the Transformer’s Positional Encoding (machine learning)
  • Corsairs3D (pirate game – arrr!)

Archive

The archive page of this blog lists all posts in chronological order.

Email Newsletter

There is an email newsletter available. Once signed up, you will receive an email every time a new post is published. That is usually about once or twice a month.

Tags

3D 3D printing Arduino Art Blender Book C++ Car Computer Science Conference Convolutional network Deep learning DenkDevelopment Ender 3 Pro Graph Hardware Haskell Historic History JavaScript Library Low-level Machine learning Math Microcontroller Miniature Modeling Music Neural network Node.js Painting Paper Philosophy Physics Piano Poker Programming Raspberry Pi SAP Science Space Telegram Telegram bot Test Tools

Navigation

Home · Contact · Legal
Proudly powered by WordPress | Theme: Sin by vsFish.

Cookies

This site uses cookies. More