![]() ![]() Suppose we have a document with the word “sky” appearing 200 times and another document with the word “sky” appearing 50, the Euclidean distance between them will be higher but the angle will still be small because they are pointing to the same direction, which is what matters when we are comparing documents. Note that even if we had a vector pointing to a point far from another vector, they still could have an small angle and that is the central point on the use of Cosine Similarity, the measurement tends to ignore the higher term count on documents. Cosine Similarity will generate a metric that says how related are two documents by looking at the angle instead of magnitude, like in the examples below: The Cosine Similarity values for different documents, 1 (same direction), 0 (90 deg.), -1 (opposite directions). What we have to do to build the cosine similarity equation is to solve the equation of the dot product for the :Īnd that is it, this is the cosine similarity formula. This metric is a measurement of orientation and not magnitude, it can be seen as a comparison between documents on a normalized space because we’re not taking into the consideration only the magnitude of each word count (tf-idf) of each document, but the angle between the documents. The cosine similarity between two vectors (or two documents on the Vector Space) is a measure that calculates the cosine of the angle between them. It is also important to note that we are using 2D examples, but the most amazing fact about it is that we can also calculate angles and similarity between vectors in higher dimensional spaces, and that is why math let us see far than the obvious even when we can’t visualize or imagine what is the angle between two vectors with twelve dimensions for instance. Now you know that, when the dot product between two different vectors is zero, they are orthogonal to each other (they have an angle of 90 degrees), this is a very neat way to check the orthogonality of different vectors. There will be no adjacent side on the triangle, it will be equivalent to zero, the term will be zero and the resulting multiplication with the magnitude of the vector will also be zero. Now, what happens when the vector is orthogonal (with an angle of 90 degrees) to the vector like on the image below ? Two orthogonal vectors (with 90 degrees angle). So, what is the term ? This term is the projection of the vector into the vector as shown on the image below: The projection of the vector A into the vector B. Rearranging the equation to understand it better using the commutative property, we have: This is all very simple and easy to understand, but what is a dot product ? What is the intuitive idea behind it ? What does it mean to have a dot product of zero ? To understand it, we need to understand what is the geometric definition of the dot product: The first thing you probably noticed is that the result of a dot product between two vectors isn’t another vector but a single value, a scalar. See an example of a dot product for two vectors with 2 dimensions each (2D): Let’s begin with the definition of the dot product for two vectors: and, where and are the components of the vector (features of the document, or TF-IDF values for each word of the document in our example) and the is the dimension of the vectors:Īs you can see, the definition of the dot product is a simple multiplication of each component from the both vectors added together. So, on the previous tutorials we learned how a document can be modeled in the Vector Space, how the TF-IDF transformation works and how the TF-IDF is calculated, now what we are going to learn is how to use a well-known similarity measure (Cosine Similarity) to calculate the similarity between different documents. Unfortunately I had no time to fix the previous tutorials for the newer versions of the scikit-learn (sklearn) package nor to answer all the questions, but I hope to do that in a close future. ![]() * It has been a long time since I wrote the TF-IDF tutorial ( Part I and Part II) and as I promissed, here is the continuation of the tutorial. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |