Commit graph

562 commits

Author SHA1 Message Date
92c7cdbf7e fix: add value annexe to META_TEXTE_VERSION.LIENS.LIEN.@naturetexte 2025-06-22 11:27:43 +02:00
0f13a08648 feat: add dry-run option to scripts that import into DB 2025-06-22 11:25:51 +02:00
cb6ebb0ce1 fix: don't fail when git set upstream fails after push 2025-06-22 09:55:40 +02:00
c2c8237fb7 fix: add value "Révision" to META_TEXTE_VERSION.LIENS.LIEN.@naturetexte 2025-06-22 08:57:20 +02:00
5ec31374c9 chore: regenerate API doc & JSON Schemas 2025-06-22 00:54:28 +02:00
1a26a08db0 fix: accept LegiSectionTa without attributes 2025-06-22 00:53:38 +02:00
33661679f1 fix: add ANNONCES to JorfSectionTaTexteNature 2025-06-21 21:35:54 +02:00
7c724472db fix: use entry.getTree instead of nodegit.Tree.lookup to have full path of entries 2025-06-21 18:16:37 +02:00
a2b8813c90 feat: use git bare repositories for database imports 2025-06-21 18:16:37 +02:00
bb3fb41778 fix: add HTTP headers when requesting DOLE HMTL pages to avoid 403 2025-06-21 18:16:37 +02:00
f40935e16e fix: typo 2025-06-21 18:16:37 +02:00
3b656dde68 chore: rename script 2025-06-21 18:16:37 +02:00
f83435949b feat: improve handling of paths in CAPP, CASS & INCA datasets 2025-06-21 18:16:37 +02:00
0777798e1d Add .xml suffix to files in liste_suppression 2025-06-21 18:16:37 +02:00
d8eccb63ef chore: improve logging 2025-06-21 18:16:37 +02:00
9145f405eb fix: Compare Oid by value instead of pointer 2025-06-21 18:16:37 +02:00
e8fc5092a0 fix: don't read modified WorkingTreeDirectory 2025-06-21 18:16:37 +02:00
e916ee24bf feat: simplify conversion of tar stream to async iterator 2025-06-21 18:16:37 +02:00
5abf7ae999 feat: download datasets using a git bare directory 2025-06-21 18:16:37 +02:00
cffcad9196 New script to generate source code fragments for metslesliens 2025-06-19 16:12:09 +02:00
57dc9d451a fix: add documentation of missing TypeScript APIs 2025-06-06 11:39:36 +02:00
51ae3f5352 feat: add generated API documentation 2025-06-06 07:24:37 +02:00
b2a1a04b91 feat: new script import_extracted_links.ts 2025-06-04 17:07:03 +02:00
0fa9b2558a chore: upgrade packages & add typedoc 2025-06-04 17:05:40 +02:00
40095aadd7 0.13.1 2025-05-28 10:46:16 +02:00
19943a564f fix: accept numbers in CONTENU; improve typings 2025-05-28 10:46:11 +02:00
2d07aab386 0.13.0 2025-05-28 09:52:07 +02:00
3fd8e3e639 fix: accept numbers in CONTENU; improve typings 2025-05-28 09:50:13 +02:00
995713f7b7 Add descriptions retrieved from DTDs to JSON Schemas 2025-05-27 22:01:05 +02:00
ef16a0f6fe Improve adding links to HTML documents 2025-05-27 21:56:41 +02:00
53dc66d59f Fix replacement of isomorphic-git with nodegit 2025-05-27 21:55:15 +02:00
google-labs-jules[bot]
401488584d feat: Update TSDoc comments to French for legal data interfaces
This commit updates the TSDoc comments in the following files to French,
as per your request:
- src/lib/legal/dole.ts
- src/lib/legal/legi.ts
- src/lib/legal/jorf.ts

The TSDoc comments were previously in English and have been re-generated
based on the Légifrance XML DTDs/XSDs, with descriptions now in French
to improve code understanding for French-speaking developers.
2025-05-27 19:43:45 +02:00
google-labs-jules[bot]
d001d67152 feat: Add TSDoc to legal data interfaces
This commit adds TSDoc comments to the TypeScript interfaces in:
- src/lib/legal/dole.ts
- src/lib/legal/legi.ts
- src/lib/legal/jorf.ts

The TSDoc comments are generated based on the information found in the
Légifrance XML DTDs (and XSDs) provided in the archive
DTD_LEGIFRANCE_20181017.7z.

These comments improve code understanding by explaining the meaning
of each interface and property, mapping them to their corresponding
XML elements and attributes as defined in the DTDs/XSDs.
2025-05-27 15:52:36 +02:00
google-labs-jules[bot]
7802dcc498 Refactor: Consolidate git helpers into nodegit directory
I've moved git-related helper files from `src/lib/server/gitify/` to `src/lib/server/nodegit/` as `isomorphic-git` has been fully replaced by `nodegit`.

- I moved contexts.ts, generators.ts, references.ts, repositories.ts, and texts_trees.ts.
- I updated all relevant import paths across the codebase to point to the new location under `nodegit`.
- I removed the now-empty `src/lib/server/gitify/` directory.
2025-05-26 21:41:58 +02:00
google-labs-jules[bot]
26964b6b7b Refactor: Further replace isomorphic-git with nodegit
I've completed the replacement of `isomorphic-git` with `nodegit` in the `src/lib/server/gitify/` directory.

Changes include:
- Updated `src/lib/server/gitify/contexts.ts` to use a plain object for tree entries instead of `isomorphic-git.TreeEntry`.
- Replaced all `isomorphic-git` function calls (`init`, `writeTree`, `writeCommit`, `writeRef`, `branch`) in `src/lib/server/gitify/generators.ts` with their `nodegit` equivalents.
2025-05-26 21:41:58 +02:00
google-labs-jules[bot]
881cb546e0 Refactor: Replace isomorphic-git with nodegit
I've replaced all usages of the `isomorphic-git` library with equivalent functionality from `nodegit`.

Changes include:
- Updated `src/lib/server/gitify/repositories.ts` to use `nodegit` for writing blobs.
- Updated `src/scripts/export_consolidated_text_to_git.ts` and `src/scripts/export_consolidated_texts_to_git.ts` to use `nodegit` for fetching the HEAD commit OID.
- Removed `isomorphic-git` from `devDependencies` in `package.json`.
2025-05-26 21:41:58 +02:00
google-labs-jules[bot]
a27868f1f4 I optimized the import scripts to reduce database writes.
I modified the four import scripts (`import_dole.ts`, `import_jorf.ts`, `import_kali.ts`, `import_legi.ts`) to conditionally update data in the PostgreSQL database.

Previously, these scripts used `INSERT ... ON CONFLICT ... DO UPDATE` for all data, which resulted in database write operations even if the incoming data was identical to the existing data.

The changes implement the following logic for each relevant table/entity within the scripts:
1. The `DO UPDATE SET` clause now uses the `EXCLUDED` keyword to refer to the values proposed for insertion.
2. A `WHERE` clause has been added to the `DO UPDATE SET` statement. This `WHERE` clause uses `IS DISTINCT FROM` to compare the current row's values with the `EXCLUDED` values (i.e., the new data).
3. The `UPDATE` is only performed if one or more of the compared fields are actually different. If the new data is identical to the existing data, the update is skipped.

This optimization significantly reduces unnecessary database write operations, especially when the input data has few or no changes compared to what's already in the database.

Minor corrections were also made to ensure `versionsRemainingElis.delete(eli)` is used consistently with `eli` as the conflict key in `import_jorf.ts` and `import_legi.ts`.
2025-05-26 15:41:20 +02:00
27cf2104ef Improve logs for unknown reference types 2025-05-01 22:52:29 +02:00
3d0165b33d Currently ignore law of ordonnance type 2025-05-01 22:51:50 +02:00
41c133526f In texts, ignore references that are not article-reference 2025-05-01 22:51:05 +02:00
8998984712 Add --default-text option to add_links_to_html_document.ts 2025-05-01 22:49:05 +02:00
edcd9d2810 Add NOTA to Markdown of LEGI articles 2025-05-01 22:47:45 +02:00
72c80244c0 Fix strings indentation 2025-05-01 22:45:01 +02:00
ad41ecac26 Add article number to Markdown when article number is in link, but not in article 2025-04-28 14:18:44 +02:00
f03127e4ce Remove wrong comment 2025-04-28 14:12:24 +02:00
cbfe1079b5 New script to add links to a HTML document using Metslesliens 2025-04-28 09:37:20 +02:00
675efff700 Slugify "’" like "'" 2025-04-28 09:34:51 +02:00
4737e2fff5 0.12.4 2025-04-26 14:18:59 +02:00
a4bc39275e Upgrade packages 2025-04-26 14:18:41 +02:00
ce800f6162 Add new nature of text: "PROJET D'AVENANT" 2025-04-26 14:11:10 +02:00