weakmap@gmail.com
265ea58bab
correct version
2022-12-11 20:08:40 +01:00
weakmap@gmail.com
ab5ee59d72
ugrading dependencies | fixing tests | supporting multiple provider of the same type
2022-12-11 20:07:18 +01:00
Christian Kellner
2062aa11a3
Scrapingant proxies ( #59 )
...
* preparing scraping ant proxies
* adding general settings for scraping ant proxy
* retrying with new ui settings
2022-06-13 08:10:30 +02:00
Christian Kellner
a4501007ff
next release version
2022-06-10 14:19:41 +02:00
Christian Kellner
bc01806421
fixing telegram provider not respecting rate limits
2022-06-10 14:19:20 +02:00
Christian Kellner
bfba6d4bd9
next release version
2022-04-29 13:26:29 +02:00
Christian Kellner
676d48807a
scraping ant retries
2022-04-29 13:22:39 +02:00
Christian Kellner
1a37773a40
Update package.json
2022-04-05 14:36:46 +02:00
Sven
67497d9828
added run-script-os to scripts to separate win use of set from unix use of export ( #52 )
...
Co-authored-by: Sven Simonsen <contact@svensimonsen.com >
2022-04-05 09:25:51 +02:00
weakmap@gmail.com
62ea296f3b
Merge branch 'master' of https://github.com/orangecoding/fredy
2022-03-27 19:43:09 +02:00
weakmap@gmail.com
52dafcef97
improving ui / ux
2022-03-27 19:42:58 +02:00
Christian Kellner
a06d20ee53
Update README.md
2022-03-26 15:09:33 +01:00
orangecoding
5347d0014d
adding scraping ant infos as we now need to use residental proxies
2022-03-25 11:19:17 +01:00
Christian Kellner
946b70003f
dependency management | fixing dev mode
2022-03-09 14:28:13 +01:00
Christian Kellner
a6e6656882
Update package.json
...
next version
2022-03-09 08:59:41 +01:00
Stefan Berger
fbea1aabc4
Add mattermost adapter ( #49 ) ( #50 )
2022-03-09 05:44:39 +01:00
orangecoding
2dd01ca38f
fixing comments
2022-02-15 13:18:20 +01:00
Carl Ambroselli
f010e8951b
Add sqlite adapter ( #48 )
2022-02-15 13:15:27 +01:00
Carl Ambroselli
5225098006
Fix address of immonet ( #47 )
2022-02-15 13:14:49 +01:00
orangecoding
6e6144e02f
fixing tests
2022-02-15 13:14:19 +01:00
Christian Kellner
aa49773a4d
improve readme
2022-01-31 10:54:12 +01:00
Jochen Schalanda
b6b8d6814c
Make requestDriver more resilient to errors ( #46 )
...
If the async request performed in `requestDriver.makeDriver()` fails, it would call the `callback` function with empty parameters but then continue the execution which can lead to the following error and crash of Fredy:
```
Error while trying to scrape data. Received error: Request failed with status code 504
/fredy/lib/services/requestDriver.js:25
if (typeof result.data === 'object' && url.toLowerCase().indexOf('scrapingant') !== -1) {
^
TypeError: Cannot read properties of undefined (reading 'data')
at driver (/fredy/lib/services/requestDriver.js:25:23)
at runMicrotasks (<anonymous>)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
```
2022-01-31 10:48:11 +01:00
Christian Kellner
b8d658a948
improve instana monitoring
2022-01-31 10:17:46 +01:00
Christian Kellner
bce0c57b02
adding instana monitoring for fredy
2022-01-31 10:01:31 +01:00
Christian Kellner
5e547baa76
Update diagram
2022-01-29 19:35:13 +01:00
Christian Kellner
b368ca7ab8
diagram.drawio
2022-01-29 19:29:02 +01:00
Christian Kellner
eb85641dfb
Update CHANGELOG.md
2022-01-28 08:50:46 +01:00
Christian Kellner
0a13037b83
Update CHANGELOG.md
...
upgrading changelog
2022-01-28 08:50:21 +01:00
Christian Kellner
5600b9766b
Update FredyRuntime.js
...
remove unnecessary todo
2022-01-26 14:48:38 +01:00
Christian Kellner
63b232521e
next version
2022-01-26 14:42:23 +01:00
Jochen Schalanda
2f5cc31ae3
Add support for Immo Südwest Presse (immo.swp.de) ( #45 )
2022-01-26 14:41:44 +01:00
Jochen Schalanda
70e78492ec
Telegram: Use job name instead of ID and link in title ( #44 )
2022-01-26 14:39:53 +01:00
Jochen Schalanda
47adb88cb5
Fix race condition if user ID is in session but not in user store ( #43 )
2022-01-25 15:11:21 +01:00
Jochen Schalanda
e5627e1d02
Allow visiting the original provider URL ( #42 )
...
Instead of truncating the original URL of each provider in the job configuration to 60 characters and losing a lot of context information, put a link to the original URL in the provider table which can be opened directly to verify what is being scraped by Fredy.
2022-01-25 14:20:42 +01:00
Christian Kellner
c831057fba
Update package.json
...
new version
2022-01-25 10:40:11 +01:00
Christian Kellner
355f3bfc76
Update README.md
...
fixing typos
2022-01-25 10:39:38 +01:00
Christian Kellner
949abcaf09
Update README.md
...
fixing badge
2022-01-25 10:38:41 +01:00
Christian Kellner
24e925ae0d
Update README.md
...
Adding new container readme, adding new test status badge
2022-01-25 10:37:49 +01:00
Jochen Schalanda
2764b2b776
Refactor Docker build and automatically push images ( #41 )
...
Refactor Docker build / Add GitHub workflow for creating and publishing Docker image
2022-01-24 16:41:23 +01:00
Jochen Schalanda
0de6d3df04
Add GitHub Actions workflow to test changes ( #40 )
2022-01-24 16:34:31 +01:00
Sebastian Wilczek
45a18529ba
Spelling and Links within Readme ( #39 )
2022-01-13 10:39:22 +01:00
orangecoding
84c8aa6d72
Merge branch 'master' of https://github.com/orangecoding/fredy
2021-12-14 20:36:16 +01:00
orangecoding
0f7a07e7fb
fixing immowelt test
2021-12-14 20:36:00 +01:00
Christian Kellner
7a65b788b9
Update issue templates
2021-12-04 13:45:20 +01:00
Christian Kellner
ec7689f73f
Update issue templates
2021-12-04 13:44:16 +01:00
orangecoding
5a13e6a0a0
next build version
2021-11-28 21:09:40 +01:00
orangecoding
7bd36e554c
fixing prod build
2021-11-28 21:09:10 +01:00
orangecoding
d7e3dfc05e
fixing version tag.. wups..
2021-11-26 21:04:16 +01:00
orangecoding
c1c4d55ede
adding or replacing sort params for provider urls when necessary
2021-11-26 21:02:09 +01:00
orangecoding
aad0884976
ignoring idea files
2021-11-25 19:16:06 +01:00
