cybertools/agent/crawl
scrat acc93c09b5 Changed bad return type self.result in outllok.py back to None
Added import for com_error type in outlook.py.

added main_outlook.py in /testing to run tests against Outlook2000. Also added outlook.cfg in /testing which has the configuration data for main_outlook.py

Removed outlook_real.txt and doctests line from tests.py

git-svn-id: svn://svn.cy55.de/Zope3/src/cybertools/trunk@2535 fd906abe-77d9-0310-91a1-e0d9ade77398
2008-04-22 21:26:14 +00:00
..
__init__.py added cybertools.agent package (work in progress...) 2008-02-23 14:07:15 +00:00
base.py intermediate commit for backup reasons 2008-04-12 13:03:09 +00:00
mail.py work in progress: fake windows API for testing with Outlook crawler 2008-04-07 07:30:47 +00:00
outlook.py Changed bad return type self.result in outllok.py back to None 2008-04-22 21:26:14 +00:00
outlook.txt work in progress: outlook crawler: make dummy implementation work 2008-04-16 16:01:39 +00:00
README.txt rearrange system startup so that components are not registered during initial import but via a controlled setup 2008-04-07 06:36:48 +00:00

================================================
Agents for Job Execution and Communication Tasks
================================================

  ($Id$)

  >>> config = '''
  ... controller(names=['core.sample'])
  ... scheduler(name='core')
  ... logger(name='default', standard=30)
  ... '''
  >>> from cybertools.agent.main import setup
  >>> master = setup(config)
  Starting agent application...
  Using controllers core.sample.


Crawler
=======

The agent uses Twisted's cooperative multitasking model.

Crawler is the base class for all derived crawlers like the filesystem crawler
and the mailcrawler. The SampleCrawler returns a deferred that already had a
callback being called, so it will return at once.

Returns a deferred that must be supplied with a callback method (and in
most cases also an errback method).

We create the sample crawler via the master's controller. The sample
controller provides a simple method for this purpose.

  >>> controller = master.controllers[0]
  >>> controller.createAgent('crawl.sample', 'crawler01')

In the next step we request the start of a job, again via the controller.

  >>> controller.enterJob('sample', 'crawler01')

The job is not executed immediately - we have to hand over control to
the twisted reactor first.

  >>> from cybertools.agent.tests import tester
  >>> tester.iterate()
  SampleCrawler is collecting.
  Job 00001 completed; result: [];