LonamiWebs/Telethon
1
1
Fork 0
mirror of https://github.com/LonamiWebs/Telethon.git synced 2024-09-21 03:08:49 +03:00
Code Issues Packages Projects Releases Wiki Activity
2,985 Commits 3 Branches 169 Tags 23 MiB
6ccd6b0a41
Commit Graph

395 Commits

Author SHA1 Message Date
Lonami Exo
67c5572d7b Possibly fix possible deadlock 2018-10-28 11:53:28 +01:00
painor
45999001be Added retry_delay parameter for auto-reconnection (#1031) 2018-10-28 10:55:58 +01:00
Lonami Exo
b0e587c03d Except expected ConnectionError in the receive loop 2018-10-26 10:36:33 +02:00
Lonami Exo
091180b32d Except CancelledError in MTProtoSender send/recv loop 
This way, the tasks should not end with the cancelled exception
which should get rid of the warning if any.
 2018-10-25 15:54:25 +02:00
Lonami Exo
4562ba9ccf Fix auth_key not actually being saved 
This would make the user have to login every time.
 2018-10-22 20:58:07 +02:00
Lonami Exo
7dece209a0 Cancel tasks on reconnect instead of awaiting them 
This prevents us from locking forever on any task that doesn't
rely on cancellation tokens, in this case, Connection.recv()'s
_recv_queue.get() would never complete after the server closed
the connection.

Additionally, working with cancellation tokens in asyncio is
somewhat annoying to do.

Last but not least removing the Connection._disconnected future
avoids the need to use its state (if an exception was set it
should be retrieved) to prevent asyncio from complaining, which
it was before.
 2018-10-21 16:20:05 +02:00
Lonami Exo
f2e77f4030 Fix using the wrong logger in Connection 2018-10-21 16:10:09 +02:00
Lonami Exo
0686ec4440 Implement after_msg_id back 2018-10-19 16:53:50 +02:00
Lonami Exo
9dc4009152 Handle the right errors 2018-10-19 14:41:50 +02:00
Lonami Exo
b93b01cb02 Fix manual BadMessageError instantiation 2018-10-19 14:02:20 +02:00
Lonami Exo
6b280dc3bd Clarify MTProtoSender._disconnected 2018-10-19 14:01:03 +02:00
Lonami Exo
945b34b103 Set MTProtoSender.auth_key on its creation 2018-10-19 13:56:11 +02:00
Lonami Exo
8563b9560d Fix auth_key not being set 2018-10-19 13:35:29 +02:00
Lonami Exo
939854a0dd Fix-up e2fe3eb (packer was OK, encryption was not made) 2018-10-19 13:33:24 +02:00
Lonami Exo
f5bc952309 Don't set the disconnected flag twice 2018-10-19 13:25:00 +02:00
Lonami Exo
e2fe3eb503 Use new broken MessagePacker 2018-10-19 13:24:52 +02:00
Lonami Exo
6d30a38316 Let Connection._disconnected be a proper Future 
This means that awaiting on disconnect will properly raise errors,
allowing to differentiate clean disconnects from faulty ones.
 2018-10-19 10:46:34 +02:00
Lonami Exo
542d0f539b Revisit connection and properly detail the class' intent 2018-10-19 10:35:22 +02:00
Lonami Exo
9cbc088b76 Make disconnect synchronous 
This also fixes a bug when auto-reconnecting in MTProtoSender.
 2018-10-16 11:56:17 +02:00
Lonami Exo
99129daeee Show the request that caused RPC errors in their messages 2018-10-15 18:35:51 +02:00
Lonami Exo
6e77f583f1 Detect arbitrary negative HTTP error codes 2018-10-12 19:47:40 +02:00
Lonami Exo
ded24db3dd Fix proxy support by removing conflicting arguments 2018-10-06 20:56:09 +02:00
Lonami Exo
fb9660afe0 Add missing awaits 2018-10-06 12:39:07 +02:00
josephbiko
3dd8b7c6d1 Support async def in sessions (#1013) 2018-10-05 20:25:49 +02:00
Lonami Exo
dc77136453 Don't expect responses from ack, log send errors, remove TODOs 2018-10-05 13:26:44 +02:00
Lonami Exo
ef60ade647 Rewrite container packing to support arbitrary sizes 2018-10-05 13:26:44 +02:00
Lonami Exo
7e7bbcf4c0 Remove irrelevant TODOs and add more logging 2018-10-05 13:26:44 +02:00
Lonami Exo
a5d4e97922 Fix alternative connection modes 2018-10-05 13:26:44 +02:00
Lonami Exo
ebde3be820 Add support for proxy again 2018-10-05 13:26:44 +02:00
Lonami Exo
db83709c6b Support connection timeout 2018-10-05 13:26:44 +02:00
Lonami Exo
1b9d6aac06 Gzip only content related data 2018-10-05 13:26:44 +02:00
Lonami Exo
37b9922f64 Handle cancellation on the receive loop 2018-10-05 13:26:44 +02:00
Lonami Exo
e319fa3aa9 Handle IncompleteReadError and InvalidChecksumError 2018-10-05 13:26:44 +02:00
Lonami Exo
3fd7c33127 Fix automatic reconnect (e.g. on bad auth key) 
This took more time than it should have to debug.
 2018-10-05 13:26:44 +02:00
Lonami Exo
bc1fd9039d Handle receiving errors 2018-10-05 13:26:44 +02:00
Lonami Exo
cf7e5d5592 Set auth_key on connection 2018-10-05 13:26:44 +02:00
Lonami Exo
21ffa2f26b Fix DC migration and seqno 2018-10-05 13:26:44 +02:00
Lonami Exo
ac567ebf1d Handle bad salt/msg not resending containers 2018-10-05 13:26:44 +02:00
Lonami Exo
3b1142aaca Add back auth key generation process 2018-10-05 13:26:44 +02:00
Lonami Exo
5edc2216c7 Handle initial connection if network is down correctly 2018-10-05 13:26:44 +02:00
Lonami Exo
2d275989cb Properly handle cancellation in _ReadyQueue 2018-10-05 13:26:44 +02:00
Lonami Exo
105bd52eee Abstract the send queue off MTProtoSender 2018-10-05 13:26:44 +02:00
Lonami Exo
b02ebcb69b Stop waiting for send items on disconnection 2018-10-05 13:26:44 +02:00
Lonami Exo
470fb9f5df Make use of the MTProtoLayer in MTProtoSender 2018-10-05 13:26:44 +02:00
Lonami Exo
9402b4a26d Create a new layer to lift encryption off the MTProtoSender 2018-10-05 13:26:44 +02:00
Lonami Exo
5daad2aaab Actually use the new connection class 2018-10-05 13:26:44 +02:00
Lonami Exo
daf94e416b Fix connection never receiving and missing clone method 2018-10-05 13:26:44 +02:00
Lonami Exo
2fd51b8582 Let all connection modes implement the new Connection 2018-10-05 13:26:44 +02:00
Lonami Exo
096424ea78 Create a new Connection class to work through queues 2018-10-05 13:26:44 +02:00
Lonami Exo
fbc46bd388 Use loop.create_future instead asyncio.Future 2018-08-21 12:22:06 +02:00
Lonami Exo
47190d7d55 Fix event loop not being passed into many asyncio calls 2018-08-21 11:31:14 +02:00
Lonami Exo
f0cd1fdd6e Don't disconnect sender on cancellation (#958) 
This was causing a "race condition" where the sender would
be disconnected after cancellation when another was created.
 2018-08-20 14:59:17 +02:00
Lonami Exo
171e13daa9 Special case File responses without parent request (#658) 2018-08-20 11:42:51 +02:00
Lonami Exo
f4b9c9d6d4 Remove critical code from assert statements 2018-08-14 19:14:13 +02:00
Lonami Exo
a688c8a4ce Fix exception ignored warnings and invalid state #935 2018-08-04 15:06:11 +02:00
Lonami Exo
7729a2a78f More logging for bad messages (#907) 2018-07-25 12:33:12 +02:00
Lonami Exo
b3990546eb Fix RPCError may occur for no parent message (#908) 2018-07-25 12:19:28 +02:00
josephbiko
e6981e7676 Remove empty except (#887) 2018-07-09 20:54:43 +02:00
Lonami Exo
7b6e65a7a5 Re-export ConnectionHttp from the network package 2018-07-08 17:48:38 +02:00
Lonami
128053750d
Implement HTTP(S) mode (closes #112) (#883) 2018-07-08 17:45:49 +02:00
Lonami Exo
393e1966c7 Avoid exceeding maximum container size 
This issue would likely be triggered when automatically
merging multiple requests into a single one while having
their size exceed 1044456 bytes like SaveFilePartRequest.

This commit avoids such issue by keeping track of the
current size, and if it exceeds the limit, avoid merge.
 2018-07-07 11:58:48 +02:00
Lonami Exo
33ce702ab9 Pre-pack outgoing TLMessage 
This has several benefits:
- The message can be resent without re-calling bytes(),
  which for some requests may be expensive.
- Converting requests to bytes early lets us detect
  errors early, such as OverflowError on bad requests.
- Containers can't exceed 1044456 bytes so knowing their
  length is important. This can now be done in O(1).

But also several drawbacks:
- If the object is modified the bytes won't reflect this.
  This isn't an issue because it's only done for in msgs.
- Incoming messages can no longer be reconverted into
  bytes but this was never needed anyway.
 2018-07-07 11:46:21 +02:00
Lonami Exo
fb40e7b508 Update mtprotoplainsender.py asserts 2018-06-28 09:48:03 +02:00
Lonami Exo
a6782ac6ea Send getState upon successful reconnection 2018-06-27 19:40:32 +02:00
Lonami Exo
18f06f1a25 Fix bad notification due to wrong system clock never ending 2018-06-27 19:04:33 +02:00
Lonami Exo
4be248932d Fix non-ending awaits due to requests not being resent on reconnect 2018-06-27 10:26:26 +02:00
Lonami Exo
58aadce5ba Fix-up 2b090f8 not actually resaving new auth_keys 2018-06-27 10:15:59 +02:00
Lonami Exo
19900a373a More useful messages from authenticator.py 2018-06-26 11:32:09 +02:00
Lonami Exo
59f6b75391 Stop showing "data left after" warning 2018-06-25 12:54:33 +02:00
Lonami Exo
184424957d Create a separate method to start reconnections 2018-06-24 10:44:31 +02:00
Lonami Exo
a99fce471a Fix cancelling a None handle on first connection+error 2018-06-21 09:48:11 +02:00
Lonami Exo
4ab5c7d92d Fix reconnect sentinel when not reconnecting in send_loop 
It would cause issues with the debug logs, since it assumes it's
a TLMessage, which have an .obj attribute. Second, the send_loop
is also smarter regarding reconnects (since OSError often occur
due to not being connected at all, namely ConnectionResetError).
 2018-06-20 18:19:57 +02:00
Lonami Exo
ad1ab64415 Save last ack to resend in case of bad salt 2018-06-20 11:12:04 +02:00
Lonami Exo
463847ad50 Revisit and remove a few TODOs 2018-06-18 18:46:06 +02:00
Lonami Exo
ebfe8ebf40 Configurable per-client request/connection retries 2018-06-18 18:13:00 +02:00
Lonami Exo
09ea1179ca Except msg_id KeyError on bad salt/msg 2018-06-18 17:20:31 +02:00
Lonami Exo
3648f7c756 Log outgoing message types and incoming message IDs 2018-06-18 17:14:04 +02:00
Lonami Exo
51d8ea0fa8 Shield disconnected Future in the MTProtoSender instead 2018-06-18 15:55:24 +02:00
Lonami Exo
7641ba2d32 Fix sleep on the wrong indent and handle CancelledError 
This was causing a huge slow-down every time a request was
received delaying all the subsequent incoming messages too
 2018-06-18 13:29:12 +02:00
Lonami Exo
374f5e2aab Periodically send ping/getState 2018-06-18 13:22:25 +02:00
Lonami Exo
94d6c339c4 Add more logging in the MTProtoSender 2018-06-18 11:29:54 +02:00
Lonami Exo
e36aff51dd Raise ConnectionError when sending without connection 
Otherwise the program could potentially block on await client(...)
since this has no timeout (unless asyncio.wait_for() was used) and
would give no further hint that the program was halted there.
 2018-06-17 20:25:22 +02:00
Dan Elkouby
d5b349e031 Implement a mechanism to notify of connection failures (#852) 2018-06-17 19:29:41 +02:00
Lonami Exo
d9d586171f Fix MTProtoSender should not send content-related queries 2018-06-17 16:23:22 +02:00
Lonami Exo
226c35ff8f Pump up default timeout from 5 to 10s 2018-06-17 11:46:56 +02:00
Lonami Exo
d18ee9ecc5 Handle (re)connection errors more gracefully 2018-06-17 11:41:35 +02:00
Lonami Exo
8086772506 Make sure not to include sentinel in containers 2018-06-16 18:40:08 +02:00
Lonami Exo
bb2cacd525 Fix automatic reconnection by unblocking the send queue 2018-06-16 18:34:36 +02:00
Dan Elkouby
df15ee421c Except connection timeouts in MTProtoSender 2018-06-15 10:58:29 +03:00
Lonami Exo
091b03c635 Use create_task instead ensure_future 2018-06-14 23:33:56 +02:00
Lonami Exo
0f14f3b16a Support custom event loops 2018-06-14 19:35:12 +02:00
Lonami Exo
908dfa148b Revisit codebase to add missing async/await 2018-06-14 17:09:20 +02:00
Lonami Exo
5bb2f50232 Handle Msg state/resend/all (from 7c0af2c by @andr-04) 2018-06-14 16:23:16 +02:00
Lonami Exo
4a9eb5b085 Handle OSError on MTProtoSender 2018-06-14 16:16:11 +02:00
Lonami Exo
8a787e90c2 Remove send/recv locks 
There is only one method sending and one method receiving,
so it doesn't make sense to lock-protect those operations.
 2018-06-13 10:04:27 +02:00
Lonami Exo
3f16c92eb3 Subclass TLRequest for content-related objects 2018-06-12 20:14:21 +02:00
Lonami Exo
d1afc70963 Fix setting Pong results 2018-06-12 19:46:37 +02:00
Lonami Exo
64dd957189 Fix None first_query and TcpClient.disconnect() 2018-06-11 19:51:01 +02:00
Lonami Exo
15ef302428 Implement _switch_dc/fix missing first request 2018-06-10 21:30:31 +02:00
Lonami Exo
4a491e45ce Fix broken debug call 2018-06-10 21:02:22 +02:00
Lonami Exo
3e151a1b7a Make TelegramBareClient able to invoke requests 2018-06-09 21:03:48 +02:00
Lonami Exo
7e68274f26 Keep consistent structure and remove done TODO 2018-06-09 15:42:10 +02:00
Lonami Exo
acd6025731 Use put_nowait and avoid double await 2018-06-09 15:26:13 +02:00
Lonami Exo
485ce5ca3b Ignore padding on server messages instead warning 
There's 12..1024 padding for the MTProto 2.0 protocol, and
the length of the message can be used to determine how much
must be read on rpc_results. However this random padding
can be safely ignored.
 2018-06-09 14:23:42 +02:00
Lonami Exo
be279ce3f5 Make TLMessage always have a valid TLObject 
This simplifies the flow instead of having separate request/body
attributes, and also means that BinaryReader.tgread_object() can
be used without so many special cases.
 2018-06-09 13:48:27 +02:00
Lonami Exo
f7e8907c6f Create RpcResult class and generalise core special cases 
This results in a cleaner MTProtoSender, which now can always
read a TLObject with a guaranteed item, if the message is OK.
 2018-06-09 13:13:55 +02:00
Lonami Exo
1e66cea9b7 Reuse some more code from MTProtoState 2018-06-09 11:36:59 +02:00
Lonami Exo
adfe861e9f Create a self-contained MTProtoState 
This frees us from using entire Session objects in something
that's supposed to just send and receive items from the net.
 2018-06-09 11:34:01 +02:00
Lonami Exo
a63580c350 Private methods are not public API 2018-06-08 21:18:15 +02:00
Lonami Exo
6766c4eea9 Make heavy use of logging 2018-06-08 21:13:14 +02:00
Lonami Exo
e36517845a Retry on connection/security errors 2018-06-08 20:50:53 +02:00
Lonami Exo
92b606a3e8 Automatically reconnect on connection reset 2018-06-08 20:42:39 +02:00
Lonami Exo
5c917fb425 Except timeout error and retry 2018-06-07 18:01:18 +02:00
Lonami Exo
f72ddbdd5a Implement retry and fail cases in authenticator 2018-06-07 17:25:02 +02:00
Lonami Exo
df895a94ab Create auth_key if not present 2018-06-07 16:32:12 +02:00
Lonami Exo
a940e2e9a2 Process entities and add a handler for updates 2018-06-07 14:32:22 +02:00
Lonami Exo
c7e4ae8672 Send acks 2018-06-07 14:16:47 +02:00
Lonami Exo
805bf00dee Support sending multiple requests at once 2018-06-07 14:04:04 +02:00
Lonami Exo
884dbe2d1f Use a custom Queue to simplify the _send_loop 2018-06-07 13:51:19 +02:00
Lonami Exo
382355a22f Collapse multiple requests into a single container 2018-06-07 13:33:32 +02:00
Lonami Exo
a3687b8bb5 Complete all methods under MTProtoSender and document them 2018-06-07 12:29:43 +02:00
Lonami Exo
56b09c0c9d Properly set future results 2018-06-07 10:30:20 +02:00
Lonami Exo
9477c75fce Fix basic requests sending and receiving 2018-06-06 21:42:48 +02:00
Lonami Exo
e469258ab9 Create a new MTProtoSender structure and its foundation 
This means that the TcpClient and the Connection (currently only
ConnectionTcpFull) will no longer be concerned about handling
errors, but the MTProtoSender will.

The foundation of the library will now be based on asyncio.
 2018-06-06 20:41:01 +02:00
Lonami Exo
ae87d452c8 Clean-up docs and merge some imports 2018-05-30 18:55:01 +02:00
Lonami Exo
ee51aa7073 Fix lost requests on disconnect need_confirmation.clear 
The pending acks shouldn't be cleared, in case of a reconnection
these would be pretty common. E.g. disconnect(), connect(),
invoke, repeat.
 2018-05-10 16:45:55 +02:00
Lonami Exo
eb22bce2d9 Add missing connect abstractmethod 2018-05-10 16:16:23 +02:00
Lonami Exo
ba4b7ce881 Make the Connection a proper ABC (#509) 2018-05-10 14:22:19 +02:00
Lonami Exo
dd954b8fbd Avoid receive busy wait when two threads receive items 2018-05-10 09:44:25 +02:00
Lonami Exo
ef509d13c7 Move InvokeAfterMsg to TLMessage to cleanly confirm results 2018-05-09 16:18:42 +02:00
Lonami Exo
e2e7e631b5 Stop using *args when invoking many requests at once 2018-05-09 10:19:45 +02:00
Lonami Exo
e200acbca8 Allow sending ordered MessageContainer 2018-05-09 09:46:07 +02:00
Lonami Exo
6652fe276c Remove broken packet length check 2018-05-07 17:05:27 +02:00
Lonami Exo
f06b9b68d5 Fix race condition causing broken responses 2018-05-07 17:01:04 +02:00
Lonami Exo
2922e8df11 Fix still broken log for broken packets 2018-05-07 16:46:58 +02:00
Lonami Exo
ace715e059 Don't log broken packets as individual bytes 2018-04-14 15:02:27 +02:00
Lonami Exo
2b9babb30f Handle GzipPacked lost requests & possibly fix reading normal 
Reading normal "lost" requests didn't .seek(-4) to read the TLObject
again. Now it has been slightly refactored to seek back always and
only seek forward when needed (e.g. rpc error).
 2018-03-30 11:50:41 +02:00
Lonami Exo
43c6896481 Add a custom role for TL references and make use of it 2018-03-23 21:42:17 +01:00
Lonami Exo
fca4904d0f Add more logging calls when confirming a request 2018-03-07 11:30:03 +01:00
Tulir Asokan
c5e6f7e265 Split Session into three parts and make a module for sessions 2018-03-01 23:40:28 +02:00
Lonami Exo
39621ceae9 Use req_pq_multi instead req_pq when creating an auth_key 2018-02-19 20:31:47 +01:00
Lonami Exo
c83638ed0e Add further logging calls to better spot lost requests 2018-02-12 13:40:00 +01:00
Lonami Exo
bf56d32118 Add missing FutureSalts response special case (#81) 2018-01-30 18:32:42 +01:00
Lonami Exo
c5e969d585 Add more useful logging on invalid packet length received 2018-01-13 19:26:45 +01:00
Lonami Exo
d81dd055e6 Remove temporary connections and use a lock again 
These seem to be the reason for missing some updates (#237)
 2018-01-06 23:45:22 +01:00
Lonami Exo
3eafe18d0b Implement MtProto 2.0 (closes #484, thanks @delivrance!) 
Huge shoutout to @delivrance's pyrogram, specially this commit:
pyrogram/pyrogram/commit/42f9a2d6994baaf9ecad590d1ff4d175a8c56454
 2018-01-06 01:55:11 +01:00
Lonami Exo
c848ae0ace Move tgread_object() outside specific msg processing calls 2017-12-20 17:45:40 +01:00
Lonami Exo
5842d3741b Make a proper use of the logging module 2017-12-20 12:47:39 +01:00
Lonami Exo
0e0bc6ecbc Fix session ID is also signed since d4d7aa9 2017-12-07 12:22:40 +01:00
Lonami Exo
1d19bb22a5 Don't ignore NewSessionCreated salt 2017-12-04 20:34:35 +01:00