Session layer problems

[Danstiv]

Checklist

• I am sure the error is coming from Pyrogram's code and not elsewhere
• I have searched in the issue tracker for similar bug reports, including closed ones
• I ran pip3 install -U https://github.com/pyrogram/pyrogram/archive/master.zip and reproduced the issue using the latest development version

Description

Hello!
I found some potential problems in the current implementation.

Ignored bad server salt

When we send ping message, the server can respond with bad server salt message, and it will be ignored because we are not waiting for the response for Ping Message, and the code that updates salt will never be reached.
This is not a big problem, because at some point Watchdog will send getUpdateState, then will receive bad server salt in response and salt will finally be updated.
But it seems that Watchdog does not maintain media sessions, which is why they probably stop working over time.

Persistent media sessions

Earlier, in version 2.0.106, sessions were created during the upload, and then closed.
At some point, media session reusing was implemented in kurigram, and perhaps this was done incorrectly.
Firstly, it is discouraged by telegram:

Downloading Files and Uploading Data to the Server
We recommend that separate connections and sessions be created for these tasks. Remember that the extra sessions must be deleted when no longer needed.

source
Secondly, As I understand it, salt in such sessions expires at some point, and if Session is not used within an hour, the server stops receiving messages from that session, and uploading / downloading files becomes impossible.

Simple fix

I think if stop ignoring salt in response to Ping, these problems will be resolved.

Complex fix

First of all, Session Layer requires refactoring and redesign, it is quite poorly designed, and this leads to such problems.
Also, the Session Pool for downloading and uploading files should implemented correctly, as recommended by Telegram.

Steps to reproduce

It is hard to reproduce, for example it is happening with my two bots, one is downloading and transcribing voice messages, other is uploading some mp3 files. At some point, any of them stop downloading or uploading files.
This has never happened in 2.0.106, the bots worked without problems for months.
I also provided some logs in #127.

Code example

Simply run that, and it will fail sooner or later.
Warning! 1 gb of free memory required!

import asyncio
import time
import os
from io import BytesIO

import pyrogram

FILE_SIZE = 2**30
file = BytesIO(os.urandom(FILE_SIZE))
file.name = 'test.bin'


last_progress_announce_time = 0
def progress(current, total):
    global last_progress_announce_time
    if time.time()-last_progress_announce_time < 5:
        return
    print(round(current/total*100))
    last_progress_announce_time = time.time()



async def main():
    client = pyrogram.Client('telegram_account', workdir='.')
    await client.start()
    for i in range(1000):
        print(f'Sending file {i+1}')
        message = await client.send_document('me', file, progress=progress)
        await message.delete()


if __name__ == '__main__':
    asyncio.run(main())

Logs

Sending file 1
0
4
8
12
14
15
17
20
22
24
26
27
29
30
31
32
33
34
37
40
43
45
46
47
49
51
52
54
56
58
60
61
62
65
69
[10] Retrying "upload.SaveBigFilePart" due to: Request timed out
[10] Retrying "upload.SaveBigFilePart" due to: Request timed out
[10] Retrying "upload.SaveBigFilePart" due to: Request timed out
[10] Retrying "upload.SaveBigFilePart" due to: Request timed out
Request timed out
Traceback (most recent call last):
  File "D:\dev\pyrogram\pyrogram\methods\advanced\save_file.py", line 109, in worker
    await session.invoke(data)
  File "D:\dev\pyrogram\pyrogram\session\session.py", line 431, in invoke
    return await self.invoke(query, retries - 1, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\dev\pyrogram\pyrogram\session\session.py", line 431, in invoke
    return await self.invoke(query, retries - 1, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\dev\pyrogram\pyrogram\session\session.py", line 431, in invoke
    return await self.invoke(query, retries - 1, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  [Previous line repeated 7 more times]
  File "D:\dev\pyrogram\pyrogram\session\session.py", line 421, in invoke
    raise e from None
  File "D:\dev\pyrogram\pyrogram\session\session.py", line 408, in invoke
    return await self.send(query, timeout=timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\dev\pyrogram\pyrogram\session\session.py", line 370, in send
    raise TimeoutError("Request timed out")
TimeoutError: Request timed out
72
Request timed out
Traceback (most recent call last):
  File "D:\dev\pyrogram\pyrogram\methods\advanced\save_file.py", line 109, in worker
    await session.invoke(data)
  File "D:\dev\pyrogram\pyrogram\session\session.py", line 431, in invoke
    return await self.invoke(query, retries - 1, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\dev\pyrogram\pyrogram\session\session.py", line 431, in invoke
    return await self.invoke(query, retries - 1, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\dev\pyrogram\pyrogram\session\session.py", line 431, in invoke
    return await self.invoke(query, retries - 1, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  [Previous line repeated 7 more times]
  File "D:\dev\pyrogram\pyrogram\session\session.py", line 421, in invoke
    raise e from None
  File "D:\dev\pyrogram\pyrogram\session\session.py", line 408, in invoke
    return await self.send(query, timeout=timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\dev\pyrogram\pyrogram\session\session.py", line 370, in send
    raise TimeoutError("Request timed out")
TimeoutError: Request timed out
Request timed out
Traceback (most recent call last):
  File "D:\dev\pyrogram\pyrogram\methods\advanced\save_file.py", line 109, in worker
    await session.invoke(data)
  File "D:\dev\pyrogram\pyrogram\session\session.py", line 431, in invoke
    return await self.invoke(query, retries - 1, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\dev\pyrogram\pyrogram\session\session.py", line 431, in invoke
    return await self.invoke(query, retries - 1, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\dev\pyrogram\pyrogram\session\session.py", line 431, in invoke
    return await self.invoke(query, retries - 1, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  [Previous line repeated 7 more times]
  File "D:\dev\pyrogram\pyrogram\session\session.py", line 421, in invoke
    raise e from None
  File "D:\dev\pyrogram\pyrogram\session\session.py", line 408, in invoke
    return await self.send(query, timeout=timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\dev\pyrogram\pyrogram\session\session.py", line 370, in send
    raise TimeoutError("Request timed out")
TimeoutError: Request timed out
Request timed out

[Danstiv]

Updated code example and logs.
@KurimuzonAkuma could you verify described problems please?

[SychO3]

I've been having this problem a lot too, so I hope it's resolved!

[SychO3]

I found it interesting that frequent use of the send file causes this issue to arise