Commit ·
b366dff
0
Parent(s):
Initial commit: repository skeleton and folder READMEs
Browse files- .gitattributes +3 -0
- .gitignore +32 -0
- LICENSE +21 -0
- README.md +32 -0
- requirements.txt +8 -0
- src/README.md +55 -0
.gitattributes
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
quran_master.db filter=lfs diff=lfs merge=lfs -text
|
| 2 |
+
extension/assets/icons/fab_logo.png filter=lfs diff=lfs merge=lfs -text
|
| 3 |
+
*.png filter=lfs diff=lfs merge=lfs -text
|
.gitignore
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Python
|
| 2 |
+
__pycache__/
|
| 3 |
+
*.py[cod]
|
| 4 |
+
*.pyo
|
| 5 |
+
*.pyd
|
| 6 |
+
|
| 7 |
+
# Environment
|
| 8 |
+
.env
|
| 9 |
+
.env.*
|
| 10 |
+
|
| 11 |
+
# Data and models (do not commit large files)
|
| 12 |
+
data/
|
| 13 |
+
models/
|
| 14 |
+
output/
|
| 15 |
+
|
| 16 |
+
# PyTorch
|
| 17 |
+
*.pt
|
| 18 |
+
*.pth
|
| 19 |
+
|
| 20 |
+
# VS Code / IDE
|
| 21 |
+
.vscode/
|
| 22 |
+
.idea/
|
| 23 |
+
|
| 24 |
+
# OS files
|
| 25 |
+
.DS_Store
|
| 26 |
+
Thumbs.db
|
| 27 |
+
|
| 28 |
+
# Jupyter
|
| 29 |
+
.ipynb_checkpoints/
|
| 30 |
+
|
| 31 |
+
# Logs
|
| 32 |
+
*.log
|
LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
MIT License
|
| 2 |
+
|
| 3 |
+
Copyright (c) 2025 Your Organization
|
| 4 |
+
|
| 5 |
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
| 6 |
+
of this software and associated documentation files (the "Software"), to deal
|
| 7 |
+
in the Software without restriction, including without limitation the rights
|
| 8 |
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
| 9 |
+
copies of the Software, and to permit persons to whom the Software is
|
| 10 |
+
furnished to do so, subject to the following conditions:
|
| 11 |
+
|
| 12 |
+
The above copyright notice and this permission notice shall be included in all
|
| 13 |
+
copies or substantial portions of the Software.
|
| 14 |
+
|
| 15 |
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
| 16 |
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
| 17 |
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
| 18 |
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
| 19 |
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 20 |
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
| 21 |
+
SOFTWARE.
|
README.md
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Arabic Grammarly (project)
|
| 2 |
+
|
| 3 |
+
Project overview
|
| 4 |
+
|
| 5 |
+
This repository contains the initial skeleton for an Arabic grammar & writing assistant (like Grammarly) project. It includes placeholders for data, models, and source code, plus configuration and dependency files. Use this repo as the single source-of-truth for team contributions.
|
| 6 |
+
|
| 7 |
+
Key features you may implement here
|
| 8 |
+
|
| 9 |
+
- Arabic spelling and grammar checking
|
| 10 |
+
- Style / clarity suggestions for Modern Standard Arabic (MSA) and dialects
|
| 11 |
+
- Sentence rewriting and paraphrasing suggestions
|
| 12 |
+
- Plagiarism or similarity checks (optional)
|
| 13 |
+
- Integration with web UI / browser extension / API
|
| 14 |
+
|
| 15 |
+
Repository layout
|
| 16 |
+
|
| 17 |
+
- `data/` — place raw and processed datasets (LANS, corpora, etc.). See `data/README.md` for details.
|
| 18 |
+
- `models/` — store training checkpoints and exports (do NOT commit large binary files). See `models/README.md`.
|
| 19 |
+
- `src/` — source code (training scripts, inference API, preprocessing). See `src/README.md`.
|
| 20 |
+
- `requirements.txt` — Python dependencies for the project.
|
| 21 |
+
- `.env.example` — template for environment variables.
|
| 22 |
+
- `.gitignore` — sensible defaults for this project.
|
| 23 |
+
|
| 24 |
+
Contributing notes
|
| 25 |
+
|
| 26 |
+
- Keep large datasets and model weights out of Git (use cloud storage or Git LFS / DVC / Hugging Face Hub).
|
| 27 |
+
- Add tests in `src/tests/` and keep the public API stable.
|
| 28 |
+
- Use small, focused pull requests that include a short description and test(s) if applicable.
|
| 29 |
+
|
| 30 |
+
Contact
|
| 31 |
+
|
| 32 |
+
If you have questions about where to add files or how to name things, ask in the team chat and follow the README inside each folder for more guidance.
|
requirements.txt
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
transformers
|
| 2 |
+
torch
|
| 3 |
+
datasets
|
| 4 |
+
camel-tools
|
| 5 |
+
sentencepiece
|
| 6 |
+
scikit-learn
|
| 7 |
+
numpy
|
| 8 |
+
pandas
|
src/README.md
ADDED
|
@@ -0,0 +1,55 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# src/ — source code
|
| 2 |
+
|
| 3 |
+
Purpose
|
| 4 |
+
|
| 5 |
+
All core project code should live in `src/`: data loaders, preprocessing, training scripts, model definitions, and inference API.
|
| 6 |
+
|
| 7 |
+
Suggested layout
|
| 8 |
+
|
| 9 |
+
- `src/app.py` — lightweight Flask/FastAPI app for inference and testing
|
| 10 |
+
- `src/train.py` — training orchestration (argument parsing, checkpointing)
|
| 11 |
+
- `src/evaluate.py` — evaluation scripts and metrics
|
| 12 |
+
- `src/data_loader.py` — dataset loaders and helpers
|
| 13 |
+
- `src/preprocessing.py` — normalization, tokenization, and text cleaning
|
| 14 |
+
- `src/models/` — model classes or adapters
|
| 15 |
+
- `src/utils/` — utility functions (logging, metrics, etc.)
|
| 16 |
+
- `src/tests/` — unit and integration tests for core modules
|
| 17 |
+
|
| 18 |
+
File responsibilities and guidelines
|
| 19 |
+
|
| 20 |
+
- Keep functions small and well-documented.
|
| 21 |
+
- Expose a stable API for inference (e.g., `predict(text: str) -> dict`). Document the input/output shapes.
|
| 22 |
+
- Add type hints where practical.
|
| 23 |
+
|
| 24 |
+
Coding conventions
|
| 25 |
+
|
| 26 |
+
- Use the project style (PEP8). Add a linter config if needed (e.g., `pyproject.toml` or `.flake8`).
|
| 27 |
+
- Write tests for data loaders and small utilities. Place tests under `src/tests/` and use pytest.
|
| 28 |
+
|
| 29 |
+
What to add when working in `src/`
|
| 30 |
+
|
| 31 |
+
- Small, focused commits that update a single feature or test.
|
| 32 |
+
- If adding a new script, update top-level `README.md` with a one-line summary and usage example.
|
| 33 |
+
- If you modify input/output shapes for the inference API, update the API docs and communicate in the PR description.
|
| 34 |
+
|
| 35 |
+
Quick example: inference contract
|
| 36 |
+
|
| 37 |
+
- Input: Arabic text string (utf-8)
|
| 38 |
+
- Output: JSON object with fields such as `text`, `suggestions` (list), `confidence` (0-1), and optional `edits` with positions
|
| 39 |
+
|
| 40 |
+
Example output shape
|
| 41 |
+
|
| 42 |
+
```
|
| 43 |
+
{
|
| 44 |
+
"text": "...",
|
| 45 |
+
"suggestions": [
|
| 46 |
+
{"start": 5, "end": 12, "replacement": "...", "explanation": "grammar: agreement"}
|
| 47 |
+
],
|
| 48 |
+
"confidence": 0.87
|
| 49 |
+
}
|
| 50 |
+
```
|
| 51 |
+
|
| 52 |
+
Testing
|
| 53 |
+
|
| 54 |
+
- Add unit tests for `preprocessing.py` and `data_loader.py` (happy path + one edge case).
|
| 55 |
+
- Add a small integration test for `app.py` that exercises the prediction endpoint with a short sample input.
|