Wikimedia Commons

File:Attention-qkv.png

Original file ‎(2,129 × 906 pixels, file size: 152 KB, MIME type: image/png)

Captions

Captions

calculation of softmax(qK)*V

Summary edit

Description
English: calculation flow through a single attention head
Date
Source Own work
Author Numiri

Licensing edit

I, the copyright holder of this work, hereby publish it under the following license:
w:en:Creative Commons
attribution share alike
This file is licensed under the Creative Commons Attribution-Share Alike 4.0 International license.
You are free:
  • to share – to copy, distribute and transmit the work
  • to remix – to adapt the work
Under the following conditions:
  • attribution – You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
  • share alike – If you remix, transform, or build upon the material, you must distribute your contributions under the same or compatible license as the original.

File history

Click on a date/time to view the file as it appeared at that time.

Date/TimeThumbnailDimensionsUserComment
current15:39, 10 April 2024Thumbnail for version as of 15:39, 10 April 20242,129 × 906 (152 KB)Numiri (talk | contribs)changed Q_w to W_q. same for K and V.
03:04, 10 May 2023Thumbnail for version as of 03:04, 10 May 20231,687 × 739 (64 KB)Numiri (talk | contribs)changed (b) to (*) since we only have one footnote item now.
02:41, 10 May 2023Thumbnail for version as of 02:41, 10 May 20231,708 × 758 (65 KB)Numiri (talk | contribs)grouped attention head, expanded softmax formula
03:17, 9 May 2023Thumbnail for version as of 03:17, 9 May 20231,506 × 712 (59 KB)Numiri (talk | contribs)changed legend labels _x_, a, b, c, d
11:16, 8 May 2023Thumbnail for version as of 11:16, 8 May 20231,517 × 705 (56 KB)Numiri (talk | contribs)Uploaded own work with UploadWizard

You cannot overwrite this file.

There are no pages that use this file.

File usage on other wikis

The following other wikis use this file:

Metadata

This file contains additional information such as Exif metadata which may have been added by the digital camera, scanner, or software program used to create or digitize it. If the file has been modified from its original state, some details such as the timestamp may not fully reflect those of the original file. The timestamp is only as accurate as the clock in the camera, and it may be completely wrong.

Retrieved from "https://commons.wikimedia.org/w/index.php?title=File:Attention-qkv.png&oldid=867234305"