weakmap@gmail.com
11a3e8771b
adding jetbrains as sponsor
2023-02-11 21:32:28 +01:00
weakmap@gmail.com
af996d81c9
smaller design improvements
2022-12-20 13:39:25 +01:00
weakmap@gmail.com
8a5fbcdf71
moving to node-fetch coz axios is causing issues with scrapingAnt
2022-12-20 10:21:15 +01:00
weakmap@gmail.com
60bb75da57
fixing dependencies
2022-12-19 21:49:47 +01:00
weakmap@gmail.com
45411080ab
adding forgotten dependencies
2022-12-19 21:48:14 +01:00
weakmap@gmail.com
4785cf797d
moving to vite as build system 🎉
2022-12-19 21:44:10 +01:00
weakmap@gmail.com
e155e992d4
Merge branch 'master' of https://github.com/orangecoding/fredy
2022-12-19 21:15:12 +01:00
weakmap@gmail.com
3ce08a3f2e
next build version
2022-12-19 21:15:01 +01:00
Christian Kellner
169655800b
Update config.json
2022-12-19 21:11:15 +01:00
weakmap@gmail.com
baf57b3641
upgrading to react 18
2022-12-19 21:10:00 +01:00
Quoc Duong Bui
47e4230b39
Provider for immobilien.de ( #65 )
...
* Add provider for immobilien.de
2022-12-19 19:29:13 +01:00
Quoc Duong Bui
c5f4333878
Update README information ( #66 )
2022-12-19 19:27:13 +01:00
Christian Kellner
c99b78fb54
Update README.md
2022-12-14 15:13:00 +01:00
Christian Kellner
88e1e1d3a9
Update README.md
2022-12-14 15:12:18 +01:00
Christian Kellner
31174b3c85
Update README.md
2022-12-14 15:11:57 +01:00
weakmap@gmail.com
265ea58bab
correct version
2022-12-11 20:08:40 +01:00
weakmap@gmail.com
ab5ee59d72
ugrading dependencies | fixing tests | supporting multiple provider of the same type
2022-12-11 20:07:18 +01:00
Christian Kellner
2062aa11a3
Scrapingant proxies ( #59 )
...
* preparing scraping ant proxies
* adding general settings for scraping ant proxy
* retrying with new ui settings
2022-06-13 08:10:30 +02:00
Christian Kellner
a4501007ff
next release version
2022-06-10 14:19:41 +02:00
Christian Kellner
bc01806421
fixing telegram provider not respecting rate limits
2022-06-10 14:19:20 +02:00
Christian Kellner
bfba6d4bd9
next release version
2022-04-29 13:26:29 +02:00
Christian Kellner
676d48807a
scraping ant retries
2022-04-29 13:22:39 +02:00
Christian Kellner
1a37773a40
Update package.json
2022-04-05 14:36:46 +02:00
Sven
67497d9828
added run-script-os to scripts to separate win use of set from unix use of export ( #52 )
...
Co-authored-by: Sven Simonsen <contact@svensimonsen.com >
2022-04-05 09:25:51 +02:00
weakmap@gmail.com
62ea296f3b
Merge branch 'master' of https://github.com/orangecoding/fredy
2022-03-27 19:43:09 +02:00
weakmap@gmail.com
52dafcef97
improving ui / ux
2022-03-27 19:42:58 +02:00
Christian Kellner
a06d20ee53
Update README.md
2022-03-26 15:09:33 +01:00
orangecoding
5347d0014d
adding scraping ant infos as we now need to use residental proxies
2022-03-25 11:19:17 +01:00
Christian Kellner
946b70003f
dependency management | fixing dev mode
2022-03-09 14:28:13 +01:00
Christian Kellner
a6e6656882
Update package.json
...
next version
2022-03-09 08:59:41 +01:00
Stefan Berger
fbea1aabc4
Add mattermost adapter ( #49 ) ( #50 )
2022-03-09 05:44:39 +01:00
orangecoding
2dd01ca38f
fixing comments
2022-02-15 13:18:20 +01:00
Carl Ambroselli
f010e8951b
Add sqlite adapter ( #48 )
2022-02-15 13:15:27 +01:00
Carl Ambroselli
5225098006
Fix address of immonet ( #47 )
2022-02-15 13:14:49 +01:00
orangecoding
6e6144e02f
fixing tests
2022-02-15 13:14:19 +01:00
Christian Kellner
aa49773a4d
improve readme
2022-01-31 10:54:12 +01:00
Jochen Schalanda
b6b8d6814c
Make requestDriver more resilient to errors ( #46 )
...
If the async request performed in `requestDriver.makeDriver()` fails, it would call the `callback` function with empty parameters but then continue the execution which can lead to the following error and crash of Fredy:
```
Error while trying to scrape data. Received error: Request failed with status code 504
/fredy/lib/services/requestDriver.js:25
if (typeof result.data === 'object' && url.toLowerCase().indexOf('scrapingant') !== -1) {
^
TypeError: Cannot read properties of undefined (reading 'data')
at driver (/fredy/lib/services/requestDriver.js:25:23)
at runMicrotasks (<anonymous>)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
```
2022-01-31 10:48:11 +01:00
Christian Kellner
b8d658a948
improve instana monitoring
2022-01-31 10:17:46 +01:00
Christian Kellner
bce0c57b02
adding instana monitoring for fredy
2022-01-31 10:01:31 +01:00
Christian Kellner
5e547baa76
Update diagram
2022-01-29 19:35:13 +01:00
Christian Kellner
b368ca7ab8
diagram.drawio
2022-01-29 19:29:02 +01:00
Christian Kellner
eb85641dfb
Update CHANGELOG.md
2022-01-28 08:50:46 +01:00
Christian Kellner
0a13037b83
Update CHANGELOG.md
...
upgrading changelog
2022-01-28 08:50:21 +01:00
Christian Kellner
5600b9766b
Update FredyRuntime.js
...
remove unnecessary todo
2022-01-26 14:48:38 +01:00
Christian Kellner
63b232521e
next version
2022-01-26 14:42:23 +01:00
Jochen Schalanda
2f5cc31ae3
Add support for Immo Südwest Presse (immo.swp.de) ( #45 )
2022-01-26 14:41:44 +01:00
Jochen Schalanda
70e78492ec
Telegram: Use job name instead of ID and link in title ( #44 )
2022-01-26 14:39:53 +01:00
Jochen Schalanda
47adb88cb5
Fix race condition if user ID is in session but not in user store ( #43 )
2022-01-25 15:11:21 +01:00
Jochen Schalanda
e5627e1d02
Allow visiting the original provider URL ( #42 )
...
Instead of truncating the original URL of each provider in the job configuration to 60 characters and losing a lot of context information, put a link to the original URL in the provider table which can be opened directly to verify what is being scraped by Fredy.
2022-01-25 14:20:42 +01:00
Christian Kellner
c831057fba
Update package.json
...
new version
2022-01-25 10:40:11 +01:00
