Rgoel1/Final Project - Revision history

Rgoel1: /* Conclusion / Future Work */

2025-05-07T20:10:26Z

‎Conclusion / Future Work

Rgoel1 at 20:03, 7 May 2025

2025-05-07T20:03:42Z

Rgoel1 at 20:01, 7 May 2025

2025-05-07T20:01:59Z

Rgoel1: /* An LLM-Based Approach to Generalized Linguistic Data Scraping */

2025-05-07T19:58:11Z

‎An LLM-Based Approach to Generalized Linguistic Data Scraping

Rgoel1: /* Evaluation */

2025-05-07T19:56:06Z

‎Evaluation

Rgoel1: /* An LLM-Based Approach to Generalized Linguistic Data Scraping */

2025-05-07T19:55:31Z

‎An LLM-Based Approach to Generalized Linguistic Data Scraping

Rgoel1: /* An LLM-Based Approach to Generalized Linguistic Data Scraping */

2025-05-07T19:54:57Z

‎An LLM-Based Approach to Generalized Linguistic Data Scraping

Rgoel1: /* An LLM-Based Approach to Generalized Linguistic Data Scraping */

2025-05-07T19:54:46Z

‎An LLM-Based Approach to Generalized Linguistic Data Scraping

Rgoel1: Created page with "==An LLM-Based Approach to Generalized Linguistic Data Scraping== * For my final project, I developed a (preliminary) general-use web scraper that uses a Large Language Model..."

2025-05-07T19:44:40Z

Created page with "==An LLM-Based Approach to Generalized Linguistic Data Scraping== * For my final project, I developed a (preliminary) general-use web scraper that uses a Large Language Model..."

New page

==An LLM-Based Approach to Generalized Linguistic Data Scraping==

* For my final project, I developed a (preliminary) general-use web scraper that uses a Large Language Model (LLM) to analyze the content and pagination structure of any given website containing linguistic text, and then automatically traverses the site while structuring/exporting the data as specified.

* The project aims to take the burden of having to manually write bespoke scrapers for different websites off of both academic linguists and communities looking to construct reliable corpora of their language's resources.

* GitHub Repository (more details in README.md): [https://github.com/rg-0el/llm-linguistics-scraping/]

===s===

@@ Line 15: / Line 15: @@
 * Overall, the scraper performed quite well, handling 3 wholly different kinds of sites with high accuracy
 * Apart from the direct objective of making construction of datasets easier, the model also proves advantageous in removing at least some need for post-processing/data cleaning steps (can ask for it to extract as sentences off the bat, removing extraneous punctuation, etc.)
-* The worst performing site was Wiktionary.org, likely due to to the Cyrillic script of the target language; the model had trouble extracting some lemmas/characters with high-fidelity, indicating a potential lack in the LLM's tokenizer vocabulary/exposure to non-Western content in training data; future work could entail finetuning an OSS model with expanded vocabulary
+* The worst performing site was Wiktionary.org, likely due to the Cyrillic script of the target language; the model had trouble extracting some lemmas/characters with high-fidelity, indicating a potential lack in the LLM's tokenizer vocabulary/exposure to non-Western content in training data; future work could entail finetuning an OSS model with expanded vocabulary
 * Potential future features include: allowing user to dictate what the structure of outputs so more information can be scraped according to the content on a given site (trivial), bettered structure detection/retry logic (smaller models are more frequently confused about page/site structures than big models; can potentially implement multi-model architecture to use a big/frontier model to first lay out a structured plan/hierarchy of the site, and then employ a smaller model to take on the actual scraping work)
 [[Category:Sp25_FinalProjects]]

← Older revision		Revision as of 20:03, 7 May 2025
Line 17:		Line 17:
	* The worst performing site was Wiktionary.org, likely due to to the Cyrillic script of the target language; the model had trouble extracting some lemmas/characters with high-fidelity, indicating a potential lack in the LLM's tokenizer vocabulary/exposure to non-Western content in training data; future work could entail finetuning an OSS model with expanded vocabulary		* The worst performing site was Wiktionary.org, likely due to to the Cyrillic script of the target language; the model had trouble extracting some lemmas/characters with high-fidelity, indicating a potential lack in the LLM's tokenizer vocabulary/exposure to non-Western content in training data; future work could entail finetuning an OSS model with expanded vocabulary
	* Potential future features include: allowing user to dictate what the structure of outputs so more information can be scraped according to the content on a given site (trivial), bettered structure detection/retry logic (smaller models are more frequently confused about page/site structures than big models; can potentially implement multi-model architecture to use a big/frontier model to first lay out a structured plan/hierarchy of the site, and then employ a smaller model to take on the actual scraping work)		* Potential future features include: allowing user to dictate what the structure of outputs so more information can be scraped according to the content on a given site (trivial), bettered structure detection/retry logic (smaller models are more frequently confused about page/site structures than big models; can potentially implement multi-model architecture to use a big/frontier model to first lay out a structured plan/hierarchy of the site, and then employ a smaller model to take on the actual scraping work)
		+
		+	[[Category:Sp25_FinalProjects]]

@@ Line 16: / Line 16: @@
 * Apart from the direct objective of making construction of datasets easier, the model also proves advantageous in removing at least some need for post-processing/data cleaning steps (can ask for it to extract as sentences off the bat, removing extraneous punctuation, etc.)
 * The worst performing site was Wiktionary.org, likely due to to the Cyrillic script of the target language; the model had trouble extracting some lemmas/characters with high-fidelity, indicating a potential lack in the LLM's tokenizer vocabulary/exposure to non-Western content in training data; future work could entail finetuning an OSS model with expanded vocabulary
-* Potential future features include: allowing user to dictate what the structure of outputs so more information can be scraped according to the content on a given site (trivial)
+* Potential future features include: allowing user to dictate what the structure of outputs so more information can be scraped according to the content on a given site (trivial), bettered structure detection/retry logic (smaller models are more frequently confused about page/site structures than big models; can potentially implement multi-model architecture to use a big/frontier model to first lay out a structured plan/hierarchy of the site, and then employ a smaller model to take on the actual scraping work)

@@ Line 8: / Line 8: @@
 ===Evaluation===
-** Bible.is (sentences from Genesis, Magyar): extracted 99.74% of all sentences without error, 4/1533 of sentences were either missing or contained a typographical error; used [existing scraper](https://github.swarthmore.edu/Ling073-sp25/web-scrapers/blob/master/bible_is_scraper.py) as ground truth/reference for accuracy calculations
+** Bible.is (sentences from Genesis, Magyar): extracted 99.74% of all sentences without error, 4/1533 of sentences were either missing or contained a typographical error; used existing scraper (https://github.swarthmore.edu/Ling073-sp25/web-scrapers/blob/master/bible_is_scraper.py) as ground truth/reference for accuracy calculations
 ** Webonary.org (all entries of dictionary, Rungus): since no bespoke/manually written scraper existed, I had to manually construct from the site a sample of 109 dictionary entries; of these, the scraper accurately extracted 97.24%, with 3/109 entries either missing or containing a typographical error
-** Wiktionary.org (lemmas, Adyghe): extracted 89.55% of all lemmas without error, 626/5624 of lemmas were either missing or contained a typographical error (see why in Future Work section); used [[existing scraper] https://github.swarthmore.edu/Ling073-sp25/web-scrapers/blob/master/wiktionary2bidix.py] as ground truth/reference for accuracy calculations
+** Wiktionary.org (lemmas, Adyghe): extracted 89.55% of all lemmas without error, 626/5624 of lemmas were either missing or contained a typographical error (see why in Future Work section); used existing scraper (https://github.swarthmore.edu/Ling073-sp25/web-scrapers/blob/master/wiktionary2bidix.py) as ground truth/reference for accuracy calculations
 ===Conclusion / Future Work===
 * Overall, the scraper performed quite well, handling 3 wholly different kinds of sites with high accuracy
 * The worst performing site was Wiktionary.org, likely due to to the Cyrillic script of the target language; the model had trouble extracting some lemmas/characters with high-fidelity, indicating a potential lack in the LLM's tokenizer vocabulary/exposure to non-Western content in training data; future work could entail finetuning an OSS model with expanded vocabulary
 * Potential future features include: allowing user to dictate what the structure of outputs so more information can be scraped according to the content on a given site (trivial)

@@ Line 8: / Line 8: @@
 ===Evaluation===
-* So far, the scraper has been tested on three different websites: Bible.is, Webonary.org, and Wiktionary.org. Here is a rundown of its performance:
+** Bible.is (sentences from Genesis, Magyar): extracted 99.74% of all sentences without error, 4/1533 of sentences were either missing or contained a typographical error; used [existing scraper](https://github.swarthmore.edu/Ling073-sp25/web-scrapers/blob/master/bible_is_scraper.py) as ground truth/reference for accuracy calculations
-  * **Bible.is** (sentences from Genesis, Magyar): extracted 99.74% of all sentences without error, 4/1533 of sentences were either missing or contained a typographical error; used [existing scraper](https://github.swarthmore.edu/Ling073-sp25/web-scrapers/blob/master/bible_is_scraper.py) as ground truth/reference for accuracy calculations
+** Webonary.org (all entries of dictionary, Rungus): since no bespoke/manually written scraper existed, I had to manually construct from the site a sample of 109 dictionary entries; of these, the scraper accurately extracted 97.24%, with 3/109 entries either missing or containing a typographical error
-  * **Webonary.org** (all entries of dictionary, Rungus): since no bespoke/manually written scraper existed, I had to manually construct from the site a sample of 109 dictionary entries; of these, the scraper accurately extracted 97.24%, with 3/109 entries either missing or containing a typographical error
+** Wiktionary.org (lemmas, Adyghe): extracted 89.55% of all lemmas without error, 626/5624 of lemmas were either missing or contained a typographical error (see why in Future Work section); used [[existing scraper] https://github.swarthmore.edu/Ling073-sp25/web-scrapers/blob/master/wiktionary2bidix.py] as ground truth/reference for accuracy calculations
 ===Conclusion / Future Work===

@@ Line 1: / Line 1: @@
 ==An LLM-Based Approach to Generalized Linguistic Data Scraping==
+===Overview===
 * For my final project, I developed a (preliminary) general-use web scraper that uses a Large Language Model (LLM) to analyze the content and pagination structure of any given website containing linguistic text, and then automatically traverses the site while structuring/exporting the data as specified.
@@ Line 7: / Line 7: @@
 * GitHub Repository (more details in README.md): [https://github.com/rg-0el/llm-linguistics-scraping/]
-===s===
+===Evaluation===