how do i interpret the results
I'm using the default policies and got the following results. How should i interpret the results? if i understand correctly, the logits tensor shape (3,2) corresponds to 3 default policies and every police has two values yes and no. so my question is why do i always get two [nan, nan]?
ShieldGemma2ImageClassifierOutputWithNoAttention(loss=None, logits=tensor([[ nan, nan],
[ nan, nan],
[35.1562, 53.1972]], device='cuda:0'), hidden_states=None, probabilities=tensor([[ nan, nan],
[ nan, nan],
[1.4618e-08, 1.0000e+00]], device='cuda:0'))
Use Latest Version Transformers，I get results like this: tensor([[5.3998e-02, 9.4600e-01],
[3.6518e-01, 6.3482e-01],
[1.8238e-08, 1.0000e+00]])，The probability of looking pornographic, dangerous, bloody and violent
Hello, the policy outputs scores for Yes and No respectively for each policy (policy is provided by text as you put)
@BITDDD
@cuiyi0326
can you put your inference code here for me to reproduce?
Hello again, I built a notebook on how to properly use ShieldGemma 2 with custom policies: https://github.com/merveenoyan/smol-vision/blob/main/ShieldGemma_2_for_Vision_LM_Safety.ipynb
Use Latest Version Transformers，I get results like this: tensor([[5.3998e-02, 9.4600e-01],
[3.6518e-01, 6.3482e-01],
[1.8238e-08, 1.0000e+00]])，The probability of looking pornographic, dangerous, bloody and violent
Here the order is "dangerous", "sexual", "violence", but I'm not sure. I guess inputing images of the three categories is the only way to find out?
Hi @cuiyi0326 ,
Apologies for the late reply, The reason your seeing [nan, nan] for two of the policies is most likely a bug in the older version of the transformers library your are using. This is known issue, and it means the model was unable to compute a valid output for those policies, which is why the values show up as "Not a Number".
The issue has been already identified and fixed in release of Latest version transformers.
Kindly use Latest Version pip install --upgrade transformers and let us know still if you have any concerns will assist you on this.
Thank you.