Ticket #1840 (new enhancement)

Opened 12 years ago

Last modified 9 years ago

Bulk uploads preserving folder structures

Reported by: jose.jimenez@… Owned by: madarche
Priority: P2 Milestone: CPS 3.5.7
Component: CPSDocument Version: TRUNK
Severity: normal Keywords:
Cc:

Description

Hello everybody,

recently i needed to create a new portal type that uploads a zip file, and recreates the same file and directories structure.

In order to do this, i have patched the module CPSDocument.createFile

I think that it can interest to somebody.

Here is the code (diff -u):

--- createFile.py 2007-04-30 13:55:52.000000000 +0200 +++ createFileNew.py 2007-04-30 13:57:44.000000000 +0200 @@ -21,7 +21,6 @@

Create a document (attached file) for each file in the uploaded ZIP, with types according to their extensions """

-

from logging import getLogger

from Products.CMFCore.utils import getToolByName

@@ -66,59 +65,68 @@

logger.info('Bad Zip File') return 0

infolist = zipfile.infolist()

  • if not check_allowed_content_types:
  • image_type_allowed = True
  • else:
  • image_type_allowed = 'Image' in allowed_content_types

# browsing the ZIP file for info in infolist:

path = info.filename

  • # Skip folders

+ list = path.split('/') + if list[len(list) - 1] == : + list = list[0:len(list)-1] + + path_filename = list[len(list)-1] + + #if is a directory

if path[-1] == '/':

  • continue
  • # Acquiring only the filename (without the directory path)
  • path_filename = generateFileName(path.split('/')[-1])

-

  • mimetype = registry.lookupExtension(path_filename.lower())
  • if mimetype is not None:
  • mimetype = mimetype.normalized()
  • else:
  • mimetype = 'application/octet-stream'

-

  • if mimetype.startswith('image/') and image_type_allowed:
  • # use the Image portal type or fallback to File if Image is not
  • # allowed
  • ptype = 'Image'
  • field_name = 'preview'
  • else:
  • ptype = 'File'

+ ptype = 'Workspace'

field_name = 'file'

+ isFolder = True + else: + mimetype = registry.lookupExtension(path_filename.lower()).normalized() + isFolder = False + + if not isFolder: + if mimetype.startswith('image/'): + ptype = 'Image' + field_name = 'preview' + else: + ptype = 'File' + field_name = 'file' +

if check_allowed_content_types and ptype not in allowed_content_types:

continue

try:

+ container = context + if len(list) > 1: + for folder in list[0:len(list) - 1]: + container = container[folder]

file_id = context.portal_workflow.invokeFactoryFor(

  • context, ptype, path_filename)

+ container, ptype, path_filename)

except BadRequest?:

logger.info('File %s already exists', path_filename) return 0

  • file_proxy = getattr(context, file_id)

+ file_proxy = getattr(container, file_id)

file_doc = file_proxy.getEditableContent()

  • # create file to attach to document
  • data = zipfile.read(path)
  • file_to_attach = File(path_filename, path_filename, data)
  • if mimetype and file_to_attach.content_type != mimetype:
  • logger.debug('Fixing mimetype from %s to %s',
  • file_to_attach.content_type, mimetype)
  • file_to_attach.manage_changeProperties(content_type=mimetype)

-

  • doc_def = {
  • 'Title': path_filename,
  • 'Description': 'Imported File (original archive: %s)' % filename,
  • field_name: file_to_attach,
  • }

- + if not isFolder: + # create file to attach to document + data = zipfile.read(path) + file_to_attach = File(path_filename, path_filename, data) + if mimetype and file_to_attach.content_type != mimetype: + logger.debug('Fixing mimetype from %s to %s', + file_to_attach.content_type, mimetype) + file_to_attach.manage_changeProperties(content_type=mimetype) + + doc_def = { + 'Title': path_filename, + 'Description': 'Imported File (original archive: %s)' % filename, + field_name: file_to_attach, + } + else: + doc_def = { + 'Title': path_filename, + 'Description': 'Imported File (original archive: %s)' % filename, + } +

file_doc.edit(doc_def, proxy=file_proxy)

return 1

Attachments

diff_createFile.txt Download (4.2 KB) - added by jose.jimenez@… 12 years ago.
diff -u original_file my_file
createFile.py.2.diff Download (4.5 KB) - added by madarche 12 years ago.
Previous diff adapted to the trunk state

Change History

Changed 12 years ago by jose.jimenez@…

diff -u original_file my_file

comment:1 Changed 12 years ago by madarche

Related to #1839.

comment:2 Changed 12 years ago by madarche

Could you please detail exactly the purpose of your patch ?

Does this patch only add a new feature ? If so could you provide use cases ?

Is the previous and default behavior preserved ?

comment:3 Changed 12 years ago by jose.jimenez@…

Sorry for the delay.

The purpose of the patch, is to maintain the structure of the files, including folder, and subfolders.

The default behavior is preserved.

Regards

Changed 12 years ago by madarche

Previous diff adapted to the trunk state

comment:4 Changed 12 years ago by madarche

Jose, I have modified your patch a little bit so that it works with the trunk.

As I can see it the only problem remaining is that some unit tests for CPSDocument don't pass anymore :

$ bin/zopectl test -vv --keepbytecode -s Products.CPSDocument -m testCreateFile

=> Ran 6 tests with 1 failures and 4 errors

Since I'm not familiar with what createFile precisely do, I would rather not adapt the unit tests myself. So for your patch to be included it is needed that you also provide a patch for tests/testCreateFile please.

comment:5 Changed 9 years ago by gracinet

  • Summary changed from possible improvement for CPSDocument.createFile to Bulk uploads preserving folder structures

createFile has changed a lot with #2205, and is about to be renamed, but I just noticed this ticket.

Of course it'd be a good feature, but that should be either a user-level choice, or at least, depend on the surrounding document type (or both). One typically does not want subfolders in Image Galleries, for instance.

Note: See TracTickets for help on using tickets.