Package com.viglet.turing.connector.plugin.webcrawler

Class TurWCPluginProcess

java.lang.Object

com.viglet.turing.connector.plugin.webcrawler.TurWCPluginProcess

@Component
public class TurWCPluginProcess
extends Object

Field Summary

Fields

Modifier and Type	Field	Description
static final String	A_HREF
static final String	ABS_HREF
static final String	AUTHORIZATION
static final String	BASIC
static final String	JAVASCRIPT
static final String	MAILTO
static final String	TEL
static final String	WEB_CRAWLER
static final String	WILD_CARD

Constructor Summary

Constructors

Constructor	Description
TurWCPluginProcess(int timeout, String referrer, TurWCAllowUrlRepository turWCAllowUrlRepository, TurWCNotAllowUrlRepository turWCNotAllowUrlRepository, TurWCFileExtensionRepository turWCFileExtensionRepository, TurWCAttributeMappingRepository turWCAttributeMappingRepository, TurWCStartingPointRepository turWCStartingPointsRepository, TurConnectorContext turConnectorContext)

Method Summary

Modifier and Type	Method	Description
static String	getCRC32Checksum(byte[] bytes)
void	start(TurWCSource turWCSource)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

MAILTO

public static final String MAILTO

See Also:

• Constant Field Values

TEL

public static final String TEL

See Also:

• Constant Field Values

JAVASCRIPT

public static final String JAVASCRIPT

See Also:

• Constant Field Values

A_HREF

public static final String A_HREF

See Also:

• Constant Field Values

ABS_HREF

public static final String ABS_HREF

See Also:

• Constant Field Values

WILD_CARD

public static final String WILD_CARD

See Also:

• Constant Field Values

AUTHORIZATION

public static final String AUTHORIZATION

See Also:

• Constant Field Values

BASIC

public static final String BASIC

See Also:

• Constant Field Values

WEB_CRAWLER

public static final String WEB_CRAWLER

See Also:

• Constant Field Values

Constructor Details

TurWCPluginProcess

public TurWCPluginProcess(@Value("${turing.wc.timeout:5000}")
 int timeout,
 @Value("${turing.wc.referrer:https://www.google.com}")
 String referrer,
 TurWCAllowUrlRepository turWCAllowUrlRepository,
 TurWCNotAllowUrlRepository turWCNotAllowUrlRepository,
 TurWCFileExtensionRepository turWCFileExtensionRepository,
 TurWCAttributeMappingRepository turWCAttributeMappingRepository,
 TurWCStartingPointRepository turWCStartingPointsRepository,
 TurConnectorContext turConnectorContext)

Method Details

start

public void start(TurWCSource turWCSource)

getCRC32Checksum

public static String getCRC32Checksum(byte[] bytes)