Commit aac764c

committed

Update dependencyvit.py

1 parent 5edd3a3 commit aac764cCopy full SHA for aac764c

File tree

-1

lines changed

-1

lines changed

Lines changed: 2 additions & 1 deletion

Original file line number	Diff line number	Diff line change
`@@ -115,11 +115,12 @@ def forward(self, x: torch.Tensor, m: torch.Tensor) -> Tuple[torch.Tensor, torch`
`115`	`115`
`116`	`116`	`#prune_mask = attn.detach().sum(1).sum(-1)`
`117`	`117`	`#prune_mask = attn.detach().sum(1).abs().sum(-1)`
`118`		`- prune_mask = attn.detach().abs().sum((1, -1))`
	`118`	`+ #prune_mask = attn.detach().abs().sum((1, -1))`
`119`	`119`	`#prune_mask = attn.sum(1).sum(-1)`
`120`	`120`	`#prune_mask = attn.sum(1).abs().sum(-1)`
`121`	`121`	`#prune_mask = attn.abs().sum((1, -1))`
`122`	`122`	`#prune_mask = m.reshape(B, N)`
	`123`	`+ purne_mask = m.detach().reshape(B, N)`
`123`	`124`
`124`	`125`	`x = self.proj(x)`
`125`	`126`	`x = self.proj_drop(x)`

Comments

(0)