-
Couldn't load subscription status.
- Fork 3.8k
Feature add cosine proximity loss #30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature add cosine proximity loss #30
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I saw 2 different implementation. First one is
return -np.sum(y_true * y_pred, axis=-1)
or
return 1. - np.sum(y_true * y_pred, axis=-1)
Which one should we choose for our implementation? They are same in nature.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the first since it ranges between -1 and 1, similar to the cosine distance itself.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a sufficient way to do grad of cosine?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doing this from my phone, so please check for errors:
If f(x, y) = (x @ y) / (norm(x) * norm(y)), then we have
df/dy = x / (norm(x) * norm(y)) - (f(x, y) * y) / (norm(y) ** 2)
where norm is just the 2-norm
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that since cosine loss == negative cosine distance, you should multiply df/dy by -1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To generate random vector array, i set the bound from 0. to 1.
@WuZhuoran - Just ping me when this is finished and I'll take a look.
@WuZhuoran - Just ping me when this is finished and I'll take a look.
@ddbourgin Thank you. I think I need some help with the grad of cos loss and how to test grad function?
General comment: It looks like right now the documentation is copied directly from scipy.spatial.distance. This needs to be rewritten before we merge.
General comment: It looks like right now the documentation is copied directly from
scipy.spatial.distance. This needs to be rewritten before we merge.
I will update documentation at next commits. Thanks
This pull request closes #29 .
- What I did
AddCosine Proximity Loss Function.- How I did it
Refer class comments.
- How to verify it
This pull request adds a new feature to Numpy-ml. Ask @ddbourgin to take a look.