Added full* markdown support and updated README

* Although the markdown parser works perfectly,
the official Telegram client does not fully reflect it.
However, if you still think that this is a lie, go check
the markdown parser and test it yourself!

README.md

@@ -6,16 +6,27 @@ on the top of the file. Also don't forget to have a look to the original project
 The files without the previously mentioned notice are no longer part of TLSharp itself, or have enough modifications
 to make them entirely different.
 
-### Requirements
+## Requirements
+### Python modules
 This project requires the following Python modules, which can be installed by issuing `sudo -H pip3 install <module>` on a
 Linux terminal:
 - `pyaes` ([GitHub](https://github.com/ricmoo/pyaes), [package index](https://pypi.python.org/pypi/pyaes))
 
-Also, you need to obtain your both [API ID and Hash](my.telegram.org). Once you have them, head to `api/` and create a copy of
-the `settings_example` file, naming it `settings` (lowercase!). Then fill the file with the corresponding values (your `api_id`,
-`api_hash` and phone number in international format). Now it is when you're ready to go!
+### Obtaining your `API ID` and `Hash`
+1. Follow [this link](https://my.telegram.org) and login with your phone number.
+2. Click under `API Development tools`.
+3. A `Create new application` window will appear. Fill in your application details.
+There is no need to enter any `URL`, and only the first two fields (`App title` and `Short name`)
+can be changed later as long as I'm aware.
+4. Click on `Create application` at the end. Now that you have the `API ID` and `Hash`,
+head to `api/` directory and create a copy of the `settings_example` file, naming it `settings` (lowercase!).
+Then fill the file with the corresponding values (your `api_id`, `api_hash` and phone number in international format).
 
-### How to add more functions to TelegramClient
+### Running Telethon
+First of all, you need to run the `tl_generator.py` by issuing `python3 tl_generator.py`. This will generate all the
+TLObjects from the given `scheme.tl` file. When it's done, you can run `python3 main.py` to start the interactive example.
+
+## How to add more functions to TelegramClient
 As of now, you cannot call any Telegram function unless you first write it by hand under `tl/telegram_client.py`. Why?
 Every Telegram function (or _request_) work in its own way. In some, you may only be interested in a single result field,
 and in others you may need to format the result in a different way. However, a plan for the future is to be able to call
@@ -44,12 +55,12 @@ open the file and see what the result will look like. Alternatively, you can sim
 Be warned that there may be more than one different type on the results. This is due to Telegram's polymorphism,
 for example, a message may or not be empty, etc.
 
-### Plans for the future
+## Plans for the future
 If everything works well, this probably ends up being a Python package :)
 
 But as of now, and until that happens, help is highly appreciated!
 
-### Code generator limitations
+## Code generator limitations
 The current code generator is not complete, yet adding the missing features would only over-complicate an already hard-to-read code.
 Some parts of the `.tl` file _should_ be omitted, because they're "built-in" in the generated code (such as writing booleans, etc.).
 

main.py

@@ -49,6 +49,6 @@ if __name__ == '__main__':
             msg = input('Enter a message: ')
             if msg == '!q':
                 break
-            client.send_message(input_peer, msg)
+            client.send_message(input_peer, msg, markdown=True, no_web_page=True)
 
         print('Thanks for trying the interactive example! Exiting.')

parser/markdown_parser.py

@@ -0,0 +1,141 @@
+from tl.types import MessageEntityBold, MessageEntityItalic, MessageEntityCode, MessageEntityTextUrl
+
+
+def parse_message_entities(msg):
+    """Parses a message and returns the parsed message and the entities (bold, italic...).
+       Note that although markdown-like syntax is used, this does not reflect the complete specification!"""
+
+    # Store the entities here
+    entities = []
+
+    # Convert the message to a mutable list
+    msg = list(msg)
+
+    # First, let's handle all the text links in the message, so afterwards it's clean
+    # for us to get our hands dirty with the other indicators (bold, italic and fixed)
+    url_indices = [None] * 4  # start/end text index, start/end url index
+    valid_url_indices = []  # all the valid url_indices found
+    for i, c in enumerate(msg):
+        if c is '[':
+            url_indices[0] = i
+
+        # From now on, also ensure that the last item was set
+        elif c == ']' and url_indices[0]:
+            url_indices[1] = i
+
+        elif c == '(' and url_indices[1]:
+            # If the previous index (']') is not exactly before the current index ('('),
+            # then it's not a valid text link, so clear the previous state
+            if url_indices[1] != i - 1:
+                url_indices[:2] = [None] * 2
+            else:
+                url_indices[2] = i
+
+        elif c == ')' and url_indices[2]:
+            # We have succeeded to find a markdown-like text link!
+            url_indices[3] = i
+            valid_url_indices.append(url_indices[:])  # Append a copy
+            url_indices = [None] * 4
+
+    # Iterate in reverse order to clean the text from the urls
+    # (not to affect previous indices) and append MessageEntityTextUrl's
+    for i in range(len(valid_url_indices) - 1, -1, -1):
+        vui = valid_url_indices[i]
+
+        # Add 1 when slicing the message not to include the [] nor ()
+        # There is no need to subtract 1 on the later part because that index is already excluded
+        link_text = ''.join(msg[vui[0]+1:vui[1]])
+        link_url = ''.join(msg[vui[2]+1:vui[3]])
+
+        # After we have retrieved both the link text and url, replace them in the message
+        # Now we do have to add 1 to include the [] and () when deleting and replacing!
+        del msg[vui[2]:vui[3]+1]
+        msg[vui[0]:vui[1]+1] = link_text
+
+        # Finally, update the current valid index url to reflect that all the previous VUI's will be removed
+        # This is because, after the previous VUI's get done, their part of the message is removed too,
+        # hence we need to update the current VUI subtracting that removed part length
+        for prev_vui in valid_url_indices[:i]:
+            prev_vui_length = prev_vui[3] - prev_vui[2] - 1
+            displacement = prev_vui_length + len('[]()')
+            vui[0] -= displacement
+            vui[1] -= displacement
+            # No need to subtract the displacement from the URL part (indices 2 and 3)
+
+        # When calculating the length, subtract 1 again not to include the previously called ']'
+        entities.append(MessageEntityTextUrl(offset=vui[0], length=vui[1] - vui[0] - 1, url=link_url))
+
+    # After the message is clean from links, handle all the indicator flags
+    indicator_flags = {
+        '*': None,
+        '_': None,
+        '`': None
+    }
+
+    # Iterate over the list to find the indicators of entities
+    for i, c in enumerate(msg):
+        # Only perform further check if the current character is an indicator
+        if c in indicator_flags:
+            # If it is the first time we find this indicator, update its index
+            if indicator_flags[c] is None:
+                indicator_flags[c] = i
+
+            # Otherwise, it means that we found it before. Hence, the message entity *is* complete
+            else:
+                # Then we have found a new whole valid entity
+                offset = indicator_flags[c]
+                length = i - offset - 1  # Subtract -1 not to include the indicator itself
+
+                # Add the corresponding entity
+                if c == '*':
+                    entities.append(MessageEntityBold(offset=offset, length=length))
+
+                elif c == '_':
+                    entities.append(MessageEntityItalic(offset=offset, length=length))
+
+                elif c == '`':
+                    entities.append(MessageEntityCode(offset=offset, length=length))
+
+                # Clear the flag to start over with this indicator
+                indicator_flags[c] = None
+
+    # Sort the entities by their offset first
+    entities = sorted(entities, key=lambda e: e.offset)
+
+    # Now that all the entities have been found and sorted, remove
+    # their indicators from the message and update the offsets
+    for entity in entities:
+        if type(entity) is not MessageEntityTextUrl:
+            # Clean the message from the current entity's indicators
+            del msg[entity.offset + entity.length + 1]
+            del msg[entity.offset]
+
+            # Iterate over all the entities but the current
+            for subentity in [e for e in entities if e is not entity]:
+                # First case, one in one out: so*me_th_in*g.
+                # In this case, the current entity length is decreased by two,
+                # and all the subentities offset decreases 1
+                if (subentity.offset > entity.offset and
+                        subentity.offset + subentity.length < entity.offset + entity.length):
+                    entity.length -= 2
+                    subentity.offset -= 1
+
+                # Second case, both inside: so*me_th*in_g.
+                # In this case, the current entity length is decreased by one,
+                # and all the subentities offset and length decrease 1
+                elif (subentity.offset > entity.offset and
+                      subentity.offset < entity.offset + entity.length and
+                          subentity.offset + subentity.length > entity.offset + entity.length):
+                    entity.length -= 1
+                    subentity.offset -= 1
+                    subentity.length -= 1
+
+                # Third case, both outside: so*me*th_in_g.
+                # In this case, the current entity is left untouched,
+                # and all the subentities offset decreases 2
+                elif subentity.offset > entity.offset + entity.length:
+                    subentity.offset -= 2
+
+    # Finally, we can join our poor mutilated message back and return
+    msg = ''.join(msg)
+    return msg, entities

tl/__init__.py

@@ -1,9 +1,14 @@
 import os
-# Only import most stuff if the TLObjects were generated
+# Only import most stuff if the TLObjects were generated and there were no errors
 if os.path.isfile('tl/all_tlobjects.py'):
-    from .all_tlobjects import tlobjects
-    from .session import Session
-    from .mtproto_request import MTProtoRequest
-    from .telegram_client import TelegramClient
+    try:
+        from .all_tlobjects import tlobjects
+        from .session import Session
+        from .mtproto_request import MTProtoRequest
+        from .telegram_client import TelegramClient
+    except Exception:
+        print('Please fix `tl_generator.py` and run it again')
+else:
+    print('Please run `python3 tl_generator.py` first')
 del os
 from .tlobject import TLObject, TLArg

tl/telegram_client.py

@@ -1,7 +1,7 @@
 # This file is based on TLSharp
 # https://github.com/sochix/TLSharp/blob/master/TLSharp.Core/TelegramClient.cs
 import platform
-import datetime
+from parser.markdown_parser import parse_message_entities
 
 import utils
 import network.authenticator
@@ -149,9 +149,18 @@ class TelegramClient:
                  TelegramClient.find_input_peer_name(dialog.peer, result.users, result.chats))
                 for dialog in result.dialogs]
 
-    def send_message(self, input_peer, message):
+    def send_message(self, input_peer, message, markdown=False, no_web_page=False):
         """Sends a message to the given input peer"""
-        request = SendMessageRequest(input_peer, message, utils.generate_random_long())
+        if markdown:
+            msg, entities = parse_message_entities(message)
+        else:
+            msg, entities = message, []
+
+        request = SendMessageRequest(peer=input_peer,
+                                     message=msg,
+                                     random_id=utils.generate_random_long(),
+                                     entities=entities,
+                                     no_webpage=no_web_page)
 
         self.sender.send(request)
         self.sender.receive(request)

tl_generator.py

@@ -38,7 +38,7 @@ def generate_tlobjects(scheme_file):
                                'functions' if tlobject.is_function
                                else 'types')
 
-        if tlobject.namespace is not None:
+        if tlobject.namespace:
             out_dir = os.path.join(out_dir, tlobject.namespace)
 
         os.makedirs(out_dir, exist_ok=True)
@@ -188,7 +188,7 @@ def get_full_file_name(tlobject):
     """Gets the full file name for the given TLObject (tl.type.full.path)"""
 
     fullname = get_file_name(tlobject, add_extension=False)
-    if tlobject.namespace is not None:
+    if tlobject.namespace:
         fullname = '{}.{}'.format(tlobject.namespace, fullname)
 
     if tlobject.is_function:
@@ -231,7 +231,7 @@ def write_onsend_code(builder, arg, args, name=None):
         if arg.type == 'true':
             return  # Exit, since True type is never written
         else:
-            builder.writeln('if {} is not None:'.format(name))
+            builder.writeln('if {}:'.format(name))
 
     if arg.is_vector:
         builder.writeln("writer.write_int(0x1cb5c415, signed=False)  # Vector's constructor ID")
@@ -248,7 +248,7 @@ def write_onsend_code(builder, arg, args, name=None):
         builder.writeln('flags = 0')
         for flag in args:
             if flag.is_flag:
-                builder.writeln('flags |= (1 << {}) if {} is not None else 0'
+                builder.writeln('flags |= (1 << {}) if {} else 0'
                                 .format(flag.flag_index, 'self.{}'.format(flag.name)))
 
         builder.writeln('writer.write_int(flags)')