External/fredy
1
0
Fork 0
mirror of https://github.com/orangecoding/fredy.git synced 2026-06-16 12:31:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
183 Commits 5 Branches 210 Tags
bc018064218d82bd5c5a1e6338ddd8e4659cf5da
Commit Graph

64 Commits

Author SHA1 Message Date
Christian Kellner
bc01806421 fixing telegram provider not respecting rate limits 2022-06-10 14:19:20 +02:00
Christian Kellner
676d48807a scraping ant retries 2022-04-29 13:22:39 +02:00
orangecoding
5347d0014d adding scraping ant infos as we now need to use residental proxies 2022-03-25 11:19:17 +01:00
Stefan Berger
fbea1aabc4 Add mattermost adapter (#49) (#50) 2022-03-09 05:44:39 +01:00
orangecoding
2dd01ca38f fixing comments 2022-02-15 13:18:20 +01:00
Carl Ambroselli
f010e8951b Add sqlite adapter (#48) 2022-02-15 13:15:27 +01:00
Carl Ambroselli
5225098006 Fix address of immonet (#47) 2022-02-15 13:14:49 +01:00
Jochen Schalanda
b6b8d6814c Make requestDriver more resilient to errors (#46) 
If the async request performed in `requestDriver.makeDriver()` fails, it would call the `callback` function with empty parameters but then continue the execution which can lead to the following error and crash of Fredy:
```
Error while trying to scrape data. Received error: Request failed with status code 504
/fredy/lib/services/requestDriver.js:25
    if (typeof result.data === 'object' && url.toLowerCase().indexOf('scrapingant') !== -1) {
                      ^

TypeError: Cannot read properties of undefined (reading 'data')
    at driver (/fredy/lib/services/requestDriver.js:25:23)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
```
 2022-01-31 10:48:11 +01:00
Christian Kellner
5600b9766b Update FredyRuntime.js 
remove unnecessary todo
 2022-01-26 14:48:38 +01:00
Jochen Schalanda
2f5cc31ae3 Add support for Immo Südwest Presse (immo.swp.de) (#45) 2022-01-26 14:41:44 +01:00
Jochen Schalanda
70e78492ec Telegram: Use job name instead of ID and link in title (#44) 2022-01-26 14:39:53 +01:00
Jochen Schalanda
47adb88cb5 Fix race condition if user ID is in session but not in user store (#43) 2022-01-25 15:11:21 +01:00
orangecoding
c1c4d55ede adding or replacing sort params for provider urls when necessary 2021-11-26 21:02:09 +01:00
Christian Kellner
8361d9c8ff splitting telegram messages into chunks to avoid errors when message exceeds limit of 4096 chars 2021-11-12 09:14:55 +01:00
Christian Kellner
c97b323b35 fix: node would crash if axios throw an error 2021-11-11 15:31:53 +01:00
Christian Kellner
ec986e4b18 adding comma separation for MailJet & Sendgrid | fixing immowelt | upgrading dependencies (#34) 2021-10-06 10:45:13 +02:00
orangecoding
8d93581dfc do not send notifications when listings have been filtered 2021-07-05 12:01:14 +02:00
Christian Kellner
59e6d287fc adding similarity check (#29) 
* adding similarity check

* adding paging

* fixing tests

* docu

* better error handling

* fixing tests

* adjusting page limit

* fixing login screen

* cleanup

* upgrade browser list

* prevent spamming the log

* fixing tests

* removing job listings when removing a job or the user
 2021-06-28 08:52:09 +02:00
Christian Kellner
97858b7539 General settings (#28) 
adding ui for general settings | adding 'working hours' as new feature
 2021-05-30 09:37:45 +02:00
orangecoding
9f0bcbd85f fixing telegram issues 2021-05-14 10:36:17 +02:00
orangecoding
1b3a95b325 making sure fredy is not crashing if scrapingant has issues 2021-05-13 20:51:15 +02:00
orangecoding
1f6e2d3618 adding inforamtion about when the processor ran the last time 2021-05-13 20:27:42 +02:00
orangecoding
793066ef94 adding screenshots for readme | adding changelog | adding error message if scrapingant was not configured when using immoscout 2021-05-13 17:22:59 +02:00
Christian Kellner
70cab66651 Bringing back immoscout (#21) 
Bringing back immoscout support 🎉
 2021-05-11 11:25:14 +02:00
orangecoding
9726079f62 fixing notifications not being called when same adapter has been used twice 2021-03-29 10:02:55 +02:00
orangecoding
0f27429fe0 updading dependencies | fixing notification-try button 2021-03-28 14:34:32 +02:00
anbucher
7541484d1f fixed docker issues (#16) 
Fixing Docker issues
 2021-03-18 14:21:36 +01:00
orangecoding
0048e72f46 fixing mixed description of mailjet and sendgrid 2021-03-06 21:34:15 +01:00
orangecoding
a3420ab4ef showin price on ebay listings 2021-03-06 21:26:39 +01:00
orangecoding
6cb24cd57f fixing ebay kleinanzeigen test 2021-03-03 15:28:27 +01:00
orangecoding
9898b65ec8 fixing immonet and neubaukompass 2021-03-01 17:29:52 +01:00
orangecoding
4180f700b5 reduce session ttl to 8 hours 2021-01-21 16:14:47 +01:00
Christian Kellner
b2847f6834 UI (#15) 
Adding new Admin UI. Updating Fredy to V3.0.0 as it has been a large rewrite. Thanks for all contributions and help on the way!
 2021-01-21 16:09:23 +01:00
Christian Kellner
1848167eda making info api port configurable 2020-08-15 13:25:17 +02:00
Christian Kellner
1558e32a80 adding Mailjet as new adapter 🎉 2020-08-15 13:22:38 +02:00
Christian Kellner
f1a00cf3bd adding option to disable the info api 2020-07-20 16:21:11 +02:00
Christian Kellner
7912c3e50a api port number typo 2020-07-17 22:22:04 +02:00
Christian Kellner
f74e36bd66 changing rest port 2020-07-17 22:15:22 +02:00
Christian Kellner
a58b849f26 sending job key with the listings 2020-07-17 16:34:10 +02:00
Christian Kellner
ee313e401c adding new sendgrid adapter 2020-07-17 15:39:11 +02:00
Christian Kellner
d1e3d989b7 updating dependencies and fixing neubaukompass 2020-06-22 09:29:32 +02:00
Christian Kellner
200e3217a4 fixing kalaydo provider 2020-03-15 16:10:49 +01:00
Christian Kellner
c91471a0b2 remove user agent from config 2020-02-28 14:03:11 +01:00
Christian Kellner
7a8db3a990 some smaller improvements 2020-02-26 18:20:41 +01:00
Christian Kellner
770d30af95 Concurrent-Jobs (#7) 
Fredy 2.0.0 introduces the concept of search jobs. You can now configure multiple of these jobs running in the same instance of Fredy. Also a new API has been created 🎉
 2020-02-26 09:05:20 +01:00
Christian Kellner
e676812688 fixing provider and their tests 2020-02-16 13:09:15 +01:00
Christian Kellner
9106144d9c fixing neubaukompass && updating dependencies 2019-10-24 21:25:19 +02:00
Christian Kellner
9961706523 adding wg-gesucht as provider 2019-06-10 14:40:28 +02:00
Christian Kellner
f429221b2e fixing immonet provider, upgrading dependencies 2019-06-10 13:26:05 +02:00
Christian Kellner
0f71c8a32e upgrade dependencies | fixing tests 2019-03-16 13:01:35 +01:00