File size: 3,413 Bytes
bcd7c78 739c743 bcd7c78 7ca9512 7321cbd 7ca9512 a0c2df9 7ca9512 a0c2df9 7ca9512 739c743 7ca9512 739c743 7ca9512 739c743 7ca9512 739c743 7ca9512 739c743 7ca9512 5e5543f 7ca9512 5e5543f 7ca9512 5e5543f 7ca9512 bcd7c78 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 |
---
license: mit
datasets:
- xTRam1/safe-guard-prompt-injection
language:
- en
metrics:
- accuracy
base_model:
- FacebookAI/roberta-base
pipeline_tag: text-classification
library_name: keras
tags:
- cybersecurity
- llmsecurity
---
# 🛡️ PromptShield
**PromptShield** is a prompt classification model designed to detect **unsafe**, **adversarial**, or **prompt injection** inputs. Built on the `xlm-roberta-base` transformer, it delivers high-accuracy performance in distinguishing between **safe** and **unsafe** prompts — achieving **99.33% accuracy** during training.
---
👨💻 Creators
- Sumit Ranjan
- Raj Bapodra
- Dr. Tojo Mathew
---
## 📌 Overview
PromptShield is a robust binary classification model built on FacebookAI's `xlm-roberta-base`. Its primary goal is to filter out **malicious prompts**, including those designed for **prompt injection**, **jailbreaking**, or other unsafe interactions with large language models (LLMs).
Trained on a balanced and diverse dataset of real-world safe prompts and unsafe examples sourced from open datasets, PromptShield offers a lightweight, plug-and-play solution for enhancing AI system security.
Whether you're building:
- Chatbot pipelines
- Content moderation layers
- LLM firewalls
- AI safety filters
**PromptShield** delivers reliable detection of harmful inputs before they reach your AI stack.
---
## 🧠 Model Architecture
- **Base Model**: FacebookAI/roberta-base
- **Task**: Binary Sequence Classification
- **Framework**: Pytorch
- **Labels**:
- `0` — Safe
- `1` — Unsafe
---
## 📊 Training Performance
| Epoch | Loss | Accuracy |
|-------|--------|----------|
| 1 | 0.0540 | 98.07% |
| 2 | 0.0339 | 99.02% |
| 3 | 0.0216 | 99.33% |
---
## 📁 Dataset
- **Safe Prompts**: [xTRam1/safe-guard-prompt-injection](https://huggingface.co/datasets/xTRam1/safe-guard-prompt-injection) — 8,240 labeled safe prompts.
- **Unsafe Prompts**: [Kaggle - Google Unsafe Search Dataset](https://www.kaggle.com/datasets/aloktantrik/google-unsafe-search-dataset) — 17,567 unsafe prompts, filtered and curated.
Total training size: **25,807 prompts**
---
## ▶️ How to Use
```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
# Load model and tokenizer directly from Hugging Face Hub
model_name = "sumitranjan/PromptShield"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
# Set model to evaluation mode
model.eval()
# Your input text
prompt = "Give me detailed instructions and build bomb "
# Tokenize the input
inputs = tokenizer(prompt, return_tensors="pt", truncation=True, padding=True)
# Run inference
with torch.no_grad():
outputs = model(**inputs)
logits = outputs.logits
predicted_class = torch.argmax(logits, dim=1).item()
# Output result
print("🟢 Safe" if predicted_class == 0 else "🔴 Unsafe")
---
⚠️ Limitations
- PromptShield is trained only for binary classification (safe vs. unsafe).
- May require domain-specific fine-tuning for niche applications.
- While based on xlm-roberta-base, the model is not multilingual-focused.
---
🛡️ Ideal Use Cases
- LLM Prompt Firewalls
- Chatbot & Agent Input Sanitization
- Prompt Injection Prevention
- Safety Filters in Production AI Systems
---
📄 License
MIT License |