Hello, I am a beginner in GAT , and I've been studying your GATv2 code lately. I have a question while going through the code in

labml_nn/graphs/gatv2/init.py

When calculating g_sum

g_sum = g_l_repeat + g_r_repeat_interleave

You mentioned in the comments: Now we add the two tensors to get

$$ \lbrace\overrightarrow{g_{l1}} + \overrightarrow{g_{r1}}, \overrightarrow{g_{l1}} + \overrightarrow{g_{r2}}, \dots, \overrightarrow{g_{l1}} + \overrightarrow{g_{rN}}, \overrightarrow{g_{l2}} + \overrightarrow{g_{r1}}, \overrightarrow{g_{l2}} + \overrightarrow{g_{r2}}, \dots, \overrightarrow{g_{l2}} + \overrightarrow{g_{rN}}, \dots\rbrace $$

But in the previous code, g_l_repeat gets

$$ \lbrace\overrightarrow{g_{l1}}, \overrightarrow{g_{l2}}, \dots, \overrightarrow{g_{lN}}, \overrightarrow{g_{l1}}, \overrightarrow{g_{l2}}, \dots, \overrightarrow{g_{lN}}, \dots\rbrace $$

and g_r_repeat_interleave gets

$$ \lbrace\overrightarrow{g_{r1}}, \overrightarrow{g_{r1}}, \dots, \overrightarrow{g_{r1}}, \overrightarrow{g_{r2}}, \overrightarrow{g_{r2}}, \dots, \overrightarrow{g_{r2}}, \dots\rbrace $$

So I think the result of adding the two tensors should be

$$ \lbrace\overrightarrow{g_{l1}} + \overrightarrow{g_{r1}}, \overrightarrow{g_{l2}} + \overrightarrow{g_{r2}}, \dots, \overrightarrow{g_{lN}} + \overrightarrow{g_{r1}}, \overrightarrow{g_{l1}} + \overrightarrow{g_{r2}}, \overrightarrow{g_{l2}} + \overrightarrow{g_{r2}}, \dots, \overrightarrow{g_{lN}} + \overrightarrow{g_{r2}}, \dots\rbrace $$

I'm not sure whether I may have overlooked some crucial information or if there's a mismatch between your comments and the code. I would greatly appreciate it if you could help clarify my confusion. Thank you.