Dataset Collection. Fixing problems with inlining #85

Vitaly-Protasov · 2020-10-27T11:03:30Z

Solved problem with commints in #50:

Problem to extract body of one-line methods: one-line several statements in a row
public String getRemainingString() { return str.substring(index); }
Problem to inline methods with massive comments inside at the end of method declaration:

/*
        } else {
              pass
        }
*/

Problem with method invocation inside other statements: if (method_inv()). Added list of statements, being inside of which this problem can occure.
Problem with None bounds detection

KatGarmash · 2020-10-29T08:05:59Z

@Vitaly-Protasov can you please pull the latest master branch?

Vitaly-Protasov · 2020-10-29T11:02:40Z

@Vitaly-Protasov can you please pull the latest master branch?

pulled

aravij

@lyriccoder Are you capable now of this PR?

veniq/dataset_collection/augmentation.py

aravij · 2020-11-03T04:38:44Z

veniq/dataset_collection/augmentation.py

@@ -75,6 +79,55 @@ def method_body_lines(method_node: ASTNode, file_path: Path) -> Tuple[int, int]:
    return start_line, end_line


+@typing.no_type_check


We better get rid of them.

aravij · 2020-11-03T04:40:28Z

veniq/dataset_collection/augmentation.py

@@ -75,6 +79,55 @@ def method_body_lines(method_node: ASTNode, file_path: Path) -> Tuple[int, int]:
    return start_line, end_line


+@typing.no_type_check
+def check_attrs(


Those attrs are always exist for ASTNode. Is it this method where we want to distinguish from None? Then we should use Optional from typing.

aravij · 2020-11-03T04:41:09Z

veniq/dataset_collection/augmentation.py

+        ASTNodeType.SUPER_CONSTRUCTOR_INVOCATION,
+        ASTNodeType.TRY_STATEMENT
+    ]
+    if check_attrs(method_invoked) and \


Nested ifs might be joined

aravij · 2020-11-03T04:43:35Z

veniq/dataset_collection/types_identifier.py

@@ -325,8 +325,7 @@ def get_lines_of_method_body(
        line_with_declaration = lines[invocation_line - 1].split('=')
        is_var_declaration = self.is_var_declaration(lines, invocation_line)
        is_direct_return = self.is_direct_return(lines, invocation_line)
-        spaces_in_body = self.complement_spaces(body_start_line + 1, invocation_line, lines) * ' '
-
+        spaces_in_body = self.complement_spaces(body_start_line, invocation_line, lines) * ' '


We better multiply string by integer, not vice versa.

why is that?

Because it is not that obvious what is happening here. If we will see string first, we will understand quicker that result is going to be another string. For now, one can consider that this expression is integer.

…-50_5

Co-authored-by: Yaroslav Kishchenko <[email protected]>

…-50_5

aravij

Few comments. And you really better to use some formatter.

aravij · 2020-11-16T08:06:31Z

veniq/dataset_collection/validation.py

@@ -277,4 +277,4 @@ def percent_matched(dataset_range_lines, semi_range_lines):
    total_case_handled = output_df.shape[0] - failed_cases_in_SEMI_algorithm - failed_cases_in_validation_examples
    if total_case_handled > 0:
        result = matched_cases / total_case_handled
-        print(f'Matched {result}% of cases, {matched_cases} out of {total_case_handled}')
+        print(f'Matched {result}% of cases, {matched_cases} out of {total_case_handled}')


Please add line break in the end of the file.

aravij · 2020-11-16T08:07:41Z

veniq/dataset_collection/types_identifier.py

@@ -153,6 +154,7 @@ def get_lines_after_invocation(
        original_file = open(filename_in, encoding='utf-8')
        lines = list(original_file)
        lines_after_invoсation = lines[invocation_line:]
+        original_file.close()


If you want, you can use with ... as ... construction.

i just fixed the bug in some function which is not mine

aravij · 2020-11-16T08:08:09Z

veniq/dataset_collection/augmentation.py

-            except Exception:
-                continue
+            except Exception as e:
+                print(str(e))


Suggested change

print(str(e))

print(e)

it's doesn't; work with JavaLang exceptions. only str(e) works

aravij · 2020-11-16T08:09:35Z

veniq/dataset_collection/augmentation.py

+                            class_name=class_name,
+                            method_node=method_node,
+                            new_full_filename=new_full_filename,
+                            original_func=original_func)


Interesting formatting.

Suggested change

original_func=original_func)

original_func=original_func

)

test/dataset_collection/test_dataset_collection.py

aravij · 2020-11-16T08:18:35Z

test/dataset_collection/test_dataset_collection.py

@@ -329,6 +344,21 @@ def test_inline_strange_body2(self):
                open(test_filepath, encoding='utf-8') as test_ex:
            self.assertEqual(actual_file.read(), test_ex.read())

+    def test_inline_comments_at_the_end(self):


Please add some empty lines in the method body to break it into logical blocks for better understandability.

veniq/dataset_collection/augmentation.py

aravij · 2020-11-16T08:22:29Z

veniq/dataset_collection/augmentation.py

+        ASTNodeType.SUPER_CONSTRUCTOR_INVOCATION,
+        ASTNodeType.TRY_STATEMENT
+    ]
+    if method_invoked.parent is not None:


Maybe it is worth to join everything into a single if. If you gonna do this, you better add some boolean variables with meaningful name.

Co-authored-by: Yaroslav Kishchenko <[email protected]>

lyriccoder · 2020-11-17T08:10:37Z

I fixed all valuable comments, the rest of them were subjective. I left them as it is

fixed problem with comments

0cdcfe5

Vitaly-Protasov marked this pull request as draft October 27, 2020 11:03

fixed invocation inside other statements

c74b1b5

Vitaly-Protasov force-pushed the issue-50_5 branch from b7279c2 to c74b1b5 Compare October 27, 2020 15:32

Vitaly-Protasov added 2 commits October 27, 2020 18:33

Merge branch 'master' of https://github.com/cqfn/veniq into issue-50_5

12d1dbf

fixed invocation inside other statements

c95ca2c

Vitaly-Protasov force-pushed the issue-50_5 branch from 2fc5cd8 to c95ca2c Compare October 27, 2020 15:36

Vitaly-Protasov added 4 commits October 27, 2020 18:41

fixed flake and logic

08d69b5

fixed flake and logic

7bd36e5

fixed flake and logic

a94320a

added test for nesting

6826e38

Vitaly-Protasov marked this pull request as ready for review October 27, 2020 15:54

Vitaly-Protasov added 4 commits October 27, 2020 19:07

added new statements

48593db

fixed previous problem with inlining declaration var

c9a657c

fixed inling spaces

b8028f4

added more statement

cc61660

Vitaly-Protasov requested a review from lyriccoder October 28, 2020 07:05

Vitaly-Protasov added 3 commits October 29, 2020 09:36

resolved conflict

da8b094

resolved conflict

751d71b

fixed test

7936613

lyriccoder requested review from aravij and KatGarmash October 29, 2020 09:49

pulled master

f43eb24

Vitaly-Protasov added 2 commits October 29, 2020 16:34

added try

42218e6

fixed bug with none bounds

91c9432

aravij suggested changes Nov 3, 2020

View reviewed changes

lyriccoder added 2 commits November 5, 2020 09:48

ADd exception

68c40c7

Fix

f673378

lyriccoder and others added 10 commits November 5, 2020 12:22

Merge branch 'master' into issue-50_5

793cf11

Merge branch 'validation' of https://github.com/cqfn/veniq into issue…

a115b2d

…-50_5

Merge branch 'master' of https://github.com/cqfn/veniq into issue-50_5

e0800cf

Merge branch 'issue-50_5' of https://github.com/cqfn/veniq into issue…

66021b8

…-50_5

Fix some prints

17aee1d

Merge branch 'master' of https://github.com/cqfn/veniq into issue-50_5

de11b43

Update veniq/dataset_collection/augmentation.py

6bb826d

Co-authored-by: Yaroslav Kishchenko <[email protected]>

Fix comments + added tests

49ce379

Merge branch 'issue-50_5' of https://github.com/cqfn/veniq into issue…

f199815

…-50_5

Fix flake8

b9815e3

lyriccoder requested a review from aravij November 13, 2020 09:28

lyriccoder added 3 commits November 13, 2020 12:31

Fix path in integration tests

f165c0e

Fix test file

c0a4a75

Fix data \r

9f60b2f

aravij reviewed Nov 16, 2020

View reviewed changes

lyriccoder and others added 2 commits November 17, 2020 10:05

Update augmentation.py

ec4b367

Update test/dataset_collection/test_dataset_collection.py

1a349b9

Co-authored-by: Yaroslav Kishchenko <[email protected]>

lyriccoder requested a review from aravij November 17, 2020 08:10

aravij approved these changes Nov 17, 2020

View reviewed changes

lyriccoder merged commit a57dcae into master Nov 17, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dataset Collection. Fixing problems with inlining #85

Dataset Collection. Fixing problems with inlining #85

Vitaly-Protasov commented Oct 27, 2020 •

edited

Loading

KatGarmash commented Oct 29, 2020

Vitaly-Protasov commented Oct 29, 2020

aravij left a comment

aravij Nov 3, 2020

lyriccoder Nov 13, 2020

aravij Nov 3, 2020

lyriccoder Nov 13, 2020

aravij Nov 3, 2020

lyriccoder Nov 13, 2020

aravij Nov 3, 2020

KatGarmash Nov 3, 2020

aravij Nov 5, 2020

lyriccoder Nov 13, 2020

aravij left a comment

aravij Nov 16, 2020

aravij Nov 16, 2020

lyriccoder Nov 17, 2020

aravij Nov 16, 2020

lyriccoder Nov 17, 2020

aravij Nov 16, 2020

aravij Nov 16, 2020

aravij Nov 16, 2020

lyriccoder commented Nov 17, 2020

		@@ -75,6 +79,55 @@ def method_body_lines(method_node: ASTNode, file_path: Path) -> Tuple[int, int]:
		return start_line, end_line


		@typing.no_type_check

Dataset Collection. Fixing problems with inlining #85

Dataset Collection. Fixing problems with inlining #85

Conversation

Vitaly-Protasov commented Oct 27, 2020 • edited Loading

KatGarmash commented Oct 29, 2020

Vitaly-Protasov commented Oct 29, 2020

aravij left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aravij left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lyriccoder commented Nov 17, 2020

Vitaly-Protasov commented Oct 27, 2020 •

edited

Loading