cco/cybertools
3
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
463 commits 5 branches 0 tags 6.4 MiB
Commit graph

56 commits

Author SHA1 Message Date
helmutm
615e8c68bf added modules for setting up real-life test runs 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2665 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-06-04 15:47:04 +00:00
helmutm
6e0ef6fd03 work in progress: SFTP transfer 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2664 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-06-04 15:45:55 +00:00
scrat
a9b8de7b21 now the doctest *really* works with the winapi.py. 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2658 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-05-31 13:48:47 +00:00
scrat
f1967e38eb some smaller changes to make the attachment examples work 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2657 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-05-31 13:32:49 +00:00
helmutm
757d47ba12 work in progress: resource/file transfer with SFTP 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2653 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-05-31 08:58:13 +00:00
helmutm
d319203a4f work in progress: transport, esp filetransfer with sftp 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2650 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-05-30 10:18:47 +00:00
helmutm
9754f0e22d set up filetransfer module 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2649 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-05-30 09:30:06 +00:00
helmutm
2909cde81d set up filetransfer module 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2648 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-05-30 09:29:57 +00:00
scrat
228b7e9077 git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2632 fd906abe-77d9-0310-91a1-e0d9ade77398 2008-05-25 17:44:57 +00:00
scrat
8e24cbc7c5 added Attachments and Attachment classes to winapi.py to get a more comprehensive doctest. In course of this change some small glitches have been found in outlook.py and mail.py, which concerned adressing of the Attachment items in outlook.py (createResource) and mail.py where the createMetadata Method now has **kw arguments for adding the filePath as a Metadata in case the mail has an attachment. 
The outlook.txt doctest has been changed to adopt the new test.
winapi now contains only one extended example but it will get two more as soon as possible.

git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2631 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-05-25 17:02:00 +00:00
helmutm
70f92e90b0 work in progress: extend and improve commandline controller 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2613 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-05-23 10:19:25 +00:00
helmutm
740fc05308 extend cmdline controller to handle simple agent and job specifications 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2612 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-05-21 11:52:28 +00:00
scrat
5bfe771ab9 moved codepages.py and the handling of outlook dialoges to system/windows/ 
writing outlook attachments to disk with mkstemp (NamedTemporaryFile would delete file after closing the handle)

git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2602 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-05-17 10:31:02 +00:00
scrat
62330d395f Added id to codepages.py 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2590 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-05-10 15:43:36 +00:00
scrat
52d6d9348c replaced createMIMEMail with an overwritten version of createResource and also moved the collection of the Metadata to the method assembleMetadata. 
Added a new module inagent/util which contains a dictionary with all more or less often used codepages and their representation in an Outlook Mail object.jsUntil now, the attachments are only represented by thei filenames as a list named subResources.

git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2589 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-05-10 15:39:25 +00:00
helmutm
ac5013a026 more on resources: extend interface and base class 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2585 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-05-09 07:47:39 +00:00
helmutm
8582252967 work in progress: structuring of resources and metadata 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2580 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-05-08 10:12:57 +00:00
scrat
167e914055 changed outlook.py: now much more data from the outlook mails is available. Now we also can access the date the mail was sent as well when it has been received. Also data about the attachements is now available (if desired). 
Minor change to winapi was necessary in order to support the doctests with the changed outlook api.
_prop_map_get now returns a dictionary instead of a tuple.

git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2577 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-05-04 21:26:57 +00:00
helmutm
4c4ada47da add basic filesystem crawler 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2570 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-05-01 18:59:06 +00:00
scrat
3acc628b80 Changed createEmailMime and loadMailsFromFolder to include now all available elements (when no key is specified) and also inform the user if a specified key does not exist or has an invalid attribute type. 
So far this Outlook Crawler works with Outlook2007 the same way like with Outlook2000 (which unfortunately also means: no date to be retrieved so far)

git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2548 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-04-26 14:38:55 +00:00
helmutm
9c92fd07f3 work in progress: object actions, stateful resources: classification quality 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2539 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-04-23 08:42:58 +00:00
helmutm
9826f31cc2 move com_error to system.windows.api 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2536 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-04-23 08:36:53 +00:00
scrat
acc93c09b5 Changed bad return type self.result in outllok.py back to None 
Added import for com_error type in outlook.py.

added main_outlook.py in /testing to run tests against Outlook2000. Also added outlook.cfg in /testing which has the configuration data for main_outlook.py

Removed outlook_real.txt and doctests line from tests.py

git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2535 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-04-22 21:26:14 +00:00
scrat
bd1c522b20 added a doctest in tests.py for testing access to the Outlook API. Minor changes in outlook.py in order to avoid Outlook COM Exception upon call on certain MailItem Attributes (Bug in Outlook API?). 
outlook.py: loadMailsFromFolder now returns self.result instead of yield None. Otherwise nothing would be returned.
Added doctest outlook_real.txt which uses the Outlook API.

git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2534 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-04-21 15:42:58 +00:00
helmutm
1a0084e46f work in progress: outlook crawler: make dummy implementation work 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2528 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-04-16 16:01:39 +00:00
helmutm
9f7afd166f add **kw to enterJob() so that job parameters may be supplied 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2524 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-04-12 17:26:36 +00:00
helmutm
97699eede7 make crawlFolders() a generator by adding yield statements 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2523 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-04-12 17:17:50 +00:00
scrat
49860fe1ce changed structure in cybertools/agent/crawl/outlook.py 
according to loops/agent/crawl/filesystem.py
adaptations and changes in winapi.py
(Added Mail class and Items in Outlook Folder)
placed comments in proper lines of winapi what next has to be found out about Outlook Mail Objects
Doctests are ok, but iterating over loadMailsFromFolder in outlook.py has a problem with NoneType and therefor does not work yet

git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2522 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-04-12 16:29:12 +00:00
scrat
c8a881041b intermediate commit for backup reasons 
added attributes and classes to winapi.py and
made some modifications to outlook.py

git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2519 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-04-12 13:03:09 +00:00
helmutm
c998205e18 make job params accessible to agent 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2511 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-04-10 12:09:01 +00:00
scrat
104d0ac88f Added some missing api. references in outlook.py 
Added some Dummy Implementations for Outlook Crawler in /testing/winapi.py
Changed outlook.txt accordingly

git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2510 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-04-09 13:46:38 +00:00
helmutm
671e15ddd5 work in progress: fake windows API for testing with Outlook crawler 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2501 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-04-07 07:30:47 +00:00
helmutm
e4108ed7d1 rearrange system startup so that components are not registered during initial import but via a controlled setup 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2500 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-04-07 06:36:48 +00:00
scrat
5aee688a12 check-in. At the moment problem with doctests for README.txt in cybertools/agent and cybertools/agent/crawl.Outlook.txt 
Problem Nr. 1 in /agent/README.txt:
    master.config
Expected:
    controller.names = ['base.sample']
    logger.name = 'default'
    logger.standard = 30
    scheduler.name = 'sample'
Got:
    logger.name = 'default'
    logger.standard = 30
    scheduler.name = 'sample'


Problem Nr. 2 in /agent/crawl/Outlook.txt:
self.agent.setupAgents(self, [spec])
File "[..]\cybertools\agent\base\agent.py", line 85, in setupAgents
        agent.name = spec.name
    AttributeError: 'NoneType' object has no attribute 'name'


git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2499 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-04-06 17:28:30 +00:00
helmutm
770a983a7f provide main and app modules for application startup; extend TelnetController to support twistd services 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2497 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-04-04 13:18:27 +00:00
helmutm
771cf29cc2 work in progress: agent application with commandline controllers 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2496 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-04-04 08:07:22 +00:00
helmutm
8f99753b1c work in progress: set up agent application; provide simple commandline controllers 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2495 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-04-03 14:13:32 +00:00
helmutm
7da8c3b9d1 rename 'inform()' to 'notify()' 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2494 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-04-03 11:27:17 +00:00
helmutm
4d8f1e7cf1 keep state information with jobs; provide feedback to master and controller via 'inform()' methods 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2493 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-04-03 11:17:37 +00:00
helmutm
7cf1a59bae keep state information with jobs; provide feedback to master and controller via 'inform()' methods 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2492 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-04-03 10:59:51 +00:00
helmutm
be4aada1f8 provide dummy methods for mail crawler 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2491 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-04-02 16:26:18 +00:00
helmutm
8ec8258a52 minor fixes in the send()/execute() chain on the Agent base classes 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2490 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-03-31 17:49:14 +00:00
helmutm
922ede26b3 work in progress: crawling: let the doctest use the full procedure via master and controller 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2489 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-03-31 17:10:08 +00:00
scrat
c3e078ca31 /agent/crawl: added the base.py and mailcrawl.py . Also added a README.txt for the doctests and modified the tests.py in /agent to recognize the new files. 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2488 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-03-30 22:36:25 +00:00
helmutm
0b91f0de2f added a queueable agent that will only execute one job at a time 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2484 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-03-29 18:01:13 +00:00
helmutm
3f69f638d3 added a queueable agent that will only execute one job at a time 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2483 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-03-29 17:58:04 +00:00
helmutm
bbb481e237 added the twisted-based scheduler to the simple doctest sequence 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2472 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-03-24 11:13:14 +00:00
helmutm
bfe47ee6a7 added the twisted-based scheduler to the simple doctest sequence 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2471 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-03-24 11:04:54 +00:00
helmutm
8a6b84fe27 make doc test run again - keep base.schedule.Scheduler 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2470 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-03-24 09:44:44 +00:00
scrat
a866d387ff Until now just a small change: added the scheduler.py now to /agent/core/ . Small change in the schedule Method. Changed the includes according to use now the schedule.py from /core. 
git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2468 fd906abe-77d9-0310-91a1-e0d9ade77398
 2008-03-23 20:16:48 +00:00