Skip to content

[18.0][IMP] base_import_pdf_by_template: Add date_time_format field to improve all formats #1349

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
victoralmau wants to merge 1 commit into
base: 18.0
Choose a base branch
from
Open

[18.0][IMP] base_import_pdf_by_template: Add date_time_format field to improve all formats #1349

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
167 changes: 82 additions & 85 deletions base_import_pdf_by_template/README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
.. image:: https://odoo-community.org/readme-banner-image
:target: https://odoo-community.org/get-involved?utm_source=readme
:alt: Odoo Community Association

===========================
Base Import Pdf by Template
===========================
Expand All @@ -17,7 +13,7 @@ Base Import Pdf by Template
.. |badge1| image:: https://img.shields.io/badge/maturity-Beta-yellow.png
:target: https://odoo-community.org/page/development-status
:alt: Beta
.. |badge2| image:: https://img.shields.io/badge/license-AGPL--3-blue.png
.. |badge2| image:: https://img.shields.io/badge/licence-AGPL--3-blue.png
:target: http://www.gnu.org/licenses/agpl-3.0-standalone.html
:alt: License: AGPL-3
.. |badge3| image:: https://img.shields.io/badge/github-OCA%2Fedi-lightgray.png?logo=github
Expand Down Expand Up @@ -56,58 +52,59 @@ is to have the document defined with a specific structure.

Fields to consider completing on template:

- Main Model: model on which the record will be generated.
Example: purchase.order
- Child field: One2many field that will create records from
selected template. Example: Order Lines (purchase.order)
- Auto detect pattern: Define a characteristic pattern of the
document so that it recognizes that it corresponds to the
template we are creating. Need to use regular expression.
Example: (?<=ESA79935607)[Ss]\*
- Header Items: Complete this field if the template has a header
table to extract information lines. Example:
Reference,Quantity,Price
- Company: Set the company that will use the template. If it is
empty, template will apply for all companies set on the
environment.
- Main Model: model on which the record will be generated.
Example: purchase.order
- Child field: One2many field that will create records from
selected template. Example: Order Lines (purchase.order)
- Auto detect pattern: Define a characteristic pattern of the
document so that it recognizes that it corresponds to the
template we are creating. Need to use regular expression.
Example: (?<=ESA79935607)[Ss]\*
- Header Items: Complete this field if the template has a header
table to extract information lines. Example:
Reference,Quantity,Price
- Company: Set the company that will use the template. If it is
empty, template will apply for all companies set on the
environment.

1. Add new lines.

..

- Related model: When adding new line, the section where to locate
the data; "header" which, as its name indicates, refers to the
header of the document and "lines" refers to the structure of lines
or table of the document.
- Related model: When adding new line, the section where to locate
the data; "header" which, as its name indicates, refers to the
header of the document and "lines" refers to the structure of
lines or table of the document.

- Field: Map the field to be completed. Example: product
- Field: Map the field to be completed. Example: product

- Pattern: Optional field to complete. Define pattern of the document
so that it recognizes the place to get the field selected on PDF
template. Need to use regular expression. Example: ([0-9]{7})
[0-7]{1}
- Pattern: Optional field to complete. Define pattern of the
document so that it recognizes the place to get the field selected
on PDF template. Need to use regular expression. Example:
([0-9]{7}) [0-7]{1}

- Value type:
- Value type:

- Fixed: Select this value, if the field mapped will always have an
specific value and not extract the information from template. In
this case Pattern field must be empty.
- Variable: Select variable to get the information from template.
In this case, Pattern field must be completed.
- Fixed: Select this value, if the field mapped will always have
an specific value and not extract the information from
template. In this case Pattern field must be empty.
- Variable: Select variable to get the information from template.
In this case, Pattern field must be completed.

- For Value type "Variable" will appear extra fields to complete:
- For Value type "Variable" will appear extra fields to complete:

- Search value: Indicates the field by which the value obtained in
the PDF will be searched on the system.
- Search value: Indicates the field by which the value obtained in
the PDF will be searched on the system.

- Default value: If the search result is empty for the search value
option, you can set default value to create a record and not
getting error message.
- Default value: If the search result is empty for the search value
option, you can set default value to create a record and not
getting error message.

- Log distint value?: This option is useful when getting prices in
order to compare prices inside system and prices obtained from PDF.
This will create lines with prices obtained from the system but
create log on chatter to see the differences obtained from PDF.
- Log distint value?: This option is useful when getting prices in
order to compare prices inside system and prices obtained from
PDF. This will create lines with prices obtained from the system
but create log on chatter to see the differences obtained from
PDF.

Check demo data to further information.

Expand All @@ -124,47 +121,47 @@ document structure.
Known issues / Roadmap
======================

- Add operator in template lines (= or ilike)
- Simplify auto-detection (defining a text only to search the system
should search the corresponding regular expression).
- Allow compatibility with registration process created from email alias
(for purchase order for example).
- Remove error if some file is not auto-detected template, options:
boolean (default option according to system parameter) to omit error
for not found files or change process to 2 steps, auto-detect and show
lines (each one with respect to a file) with template applied (similar
to dms_auto_classification).
- Display a more readable error if there is an error in Preview process,
example: wrong pattern. Message: "Please check template defined, some
items are not correctly set".
- Add a progress bar (widget=“gauge”) in the import wizard process,
useful if we import for example sales orders with 20 lines and thus
know the progress.
- Add date_format model to define the specific formats.
- Add operator in template lines (= or ilike)
- Simplify auto-detection (defining a text only to search the system
should search the corresponding regular expression).
- Allow compatibility with registration process created from email
alias (for purchase order for example).
- Remove error if some file is not auto-detected template, options:
boolean (default option according to system parameter) to omit error
for not found files or change process to 2 steps, auto-detect and
show lines (each one with respect to a file) with template applied
(similar to dms_auto_classification).
- Display a more readable error if there is an error in Preview
process, example: wrong pattern. Message: "Please check template
defined, some items are not correctly set".
- Add a progress bar (widget=“gauge”) in the import wizard process,
useful if we import for example sales orders with 20 lines and thus
know the progress.
- Add date_format model to define the specific formats.

Compatibility with csv, xls, etc:

- Separate much of the logic to new module base_import_simple that would
contain the logic of templates, type of files (csv, excel, etc) in the
templates and wizard and this module would depend on the other adding
only what relates to PDF.
- The base module should take into account for each template whether
each line is a new record or not, and start line (in case you want to
omit any), only page 1 would be imported.
- The preview smart-btton would serve exactly the same purpose.
- In the case of csv and Excel that each record is a line, the document
will NOT be attached to the record.
- If you indicate that each record is a line the column will be the key,
otherwise you must specify to which line each line of the template
refers.
- In the case of csv it will try to auto-detect the lines and columns
(no need to complicate delimiters configuration).
- The menu "Import PDF" of the favorite menu would become "Import file",
and the allowed file extensions would be those obtained from a method
(it would be extended by other modules that add other formats such as
PDF).
- Add queue_job_base_import_simple module to process everything by
queues (example: Excel with hundreds of lines, each one a record).
- Separate much of the logic to new module base_import_simple that
would contain the logic of templates, type of files (csv, excel, etc)
in the templates and wizard and this module would depend on the other
adding only what relates to PDF.
- The base module should take into account for each template whether
each line is a new record or not, and start line (in case you want to
omit any), only page 1 would be imported.
- The preview smart-btton would serve exactly the same purpose.
- In the case of csv and Excel that each record is a line, the document
will NOT be attached to the record.
- If you indicate that each record is a line the column will be the
key, otherwise you must specify to which line each line of the
template refers.
- In the case of csv it will try to auto-detect the lines and columns
(no need to complicate delimiters configuration).
- The menu "Import PDF" of the favorite menu would become "Import
file", and the allowed file extensions would be those obtained from a
method (it would be extended by other modules that add other formats
such as PDF).
- Add queue_job_base_import_simple module to process everything by
queues (example: Excel with hundreds of lines, each one a record).

Bug Tracker
===========
Expand All @@ -187,10 +184,10 @@ Authors
Contributors
------------

- `Tecnativa <https://www.tecnativa.com>`__:
- `Tecnativa <https://www.tecnativa.com>`__:

- Víctor Martínez
- Pedro M. Baeza
- Víctor Martínez
- Pedro M. Baeza

Maintainers
-----------
Expand Down
2 changes: 1 addition & 1 deletion base_import_pdf_by_template/__manifest__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl).
{
"name": "Base Import Pdf by Template",
"version": "18.0.1.0.1",
"version": "18.0.1.1.0",
"website": "https://github.com/OCA/edi",
"author": "Tecnativa, Odoo Community Association (OCA)",
"license": "AGPL-3",
Expand Down
32 changes: 32 additions & 0 deletions base_import_pdf_by_template/migrations/18.0.1.1.0/pre-migration.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Copyright 2026 Tecnativa - Víctor Martínez
# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl).

from openupgradelib import openupgrade


@openupgrade.migrate()
def migrate(env, version):
openupgrade.logged_query(
env.cr,
"""
ALTER TABLE base_import_pdf_template_line
ADD COLUMN IF NOT EXISTS date_time_format VARCHAR
""",
)
openupgrade.logged_query(
env.cr,
"""
UPDATE base_import_pdf_template_line biptl
SET date_time_format =
replace(replace(biptl.date_format, '*', '%'), '-short', '')
||
CASE
WHEN imf.ttype = 'datetime'
THEN ' ' || replace(biptl.time_format, '*', '%')
ELSE ''
END
FROM ir_model_fields imf
WHERE biptl.field_id = imf.id
AND biptl.date_format IS NOT NULL;
""",
)
43 changes: 3 additions & 40 deletions base_import_pdf_by_template/models/base_import_pdf_template.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -239,12 +239,7 @@ class BaseImportPdfTemplateLine(models.Model):
selection="_selection_reference_value",
string="Default value",
)
date_format = fields.Selection(
selection="_selection_date_format",
)
time_format = fields.Selection(
selection="_selection_time_format",
)
date_time_format = fields.Char()
decimal_separator = fields.Selection(
selection=[
("dot", "Dot (.)"),
Expand Down Expand Up @@ -301,32 +296,6 @@ def _selection_related_model(self):
("lines", self.env._("Lines")),
]

@api.model
def _selection_date_format(self):
return [
("*Y-*d-*m", self.env._("YY-dd-MM")),
("*m-*d-*Y", self.env._("MM-dd-YY")),
("*d-*m-*Y", self.env._("dd-MM-YY")),
("*Y/*d/*m", self.env._("YY/dd/MM")),
("*m/*d/*Y", self.env._("MM/dd/YY")),
("*d.*m.*Y", self.env._("dd.MM.YY")),
("*d.*m.*y-short", self.env._("dd.MM.yy")),
("*d/*m/*Y", self.env._("dd/MM/YY")),
("*d/*m/*y-short", self.env._("dd/MM/yy")),
("*B *d, *Y", self.env._("B dd, YY")),
("*b-short *d, *Y", self.env._("b dd, YY")),
("*d *b-short *Y", self.env._("dd b YY")),
("*d *B *Y", self.env._("dd B YY")),
("*d-*b-*y", self.env._("dd-b-yy")),
("*d-*b-short-*Y", self.env._("dd-b-YY")),
]

@api.model
def _selection_time_format(self):
return [
("*H:*M:*S", self.env._("H:M:S")),
]

@api.model
def _selection_thousand_separator(self):
return [
Expand Down Expand Up @@ -397,15 +366,9 @@ def _replace_text(self, text, letters, prefix):
return text

def _process_datetime_value(self, value):
if self.field_ttype not in ("date", "datetime") or not self.date_format:
if self.field_ttype not in ("date", "datetime") or not self.date_time_format:
return value
# We need to replace -short because it is not respected in databases with
# different capitalization. (example: *d/*m/*Y and *d/*m/*y)
date_format = self.date_format.replace("*", "%").replace("-short", "")
if self.field_ttype == "datetime":
time_format = self.time_format.replace("*", "%")
date_format += " " + time_format
datetime_object = datetime.strptime(value, date_format)
datetime_object = datetime.strptime(value, self.date_time_format)
value = datetime_object.strftime("%Y-%m-%d %H:%M:%S")
return value

Expand Down
Loading
Loading