Protocole::V06::Extended

[nabilbendafi]

Please find a pull request regarding https://tools.ietf.org/html/draft-ietf-secsh-filexfer-09 [9. Extensions]

Handle SFTP V6 extensions commands

• md5-hash-handle
• check-file-handle
• space-available
• home-directory

Test results under ruby 2.3.1p112 (2016-04-26) [x86_64-linux-gnu] + net-ssh-4.1.0 follows

ruby2.3 -Itest test/protocol/06/test_extended.rb -v
Loaded suite test/protocol/06/test_extended
Started
Net::SFTP::TestCase: 
  default_test:								.: (0.002273)
  Protocol::V01::TestBase: 
    test_block_should_raise_not_implemented_error:			.: (0.000703)
    test_close_should_send_close_packet:					.: (0.000470)
    test_fsetstat_should_translate_hash_to_attributes_and_send_fsetstat_packet:.: (0.000602)
    test_fstat_should_ignore_flags_parameter:				.: (0.000428)
    test_fstat_should_send_fstat_packet:					.: (0.000404)
    test_link_should_raise_not_implemented_error:			.: (0.000323)
    test_lstat_should_ignore_flags_parameter:				.: (0.000443)
    test_lstat_should_send_lstat_packet:					.: (0.000404)
    test_mkdir_should_translate_hash_to_attributes_and_send_mkdir_packet:.: (0.000490)
    test_open_with_a_plus_should_translate_correctly:			.: (0.000516)
    test_open_with_a_should_translate_correctly:				.: (0.000462)
    test_open_with_attributes_converts_hash_to_attribute_packet:		.: (0.000463)
    test_open_with_numeric_flag_should_accept_IO_constants:		.: (0.000530)
    test_open_with_r_plus_should_translate_correctly:			.: (0.000488)
    test_open_with_r_should_translate_correctly:				.: (0.000527)
    test_open_with_rb_should_translate_correctly:			.: (0.000527)
    test_open_with_w_plus_should_translate_correctly:			.: (0.000552)
    test_open_with_w_should_translate_correctly:				.: (0.000474)
    test_opendir_should_send_opendir_packet:				.: (0.000458)
    test_parse_attrs_packet_should_use_correct_attributes_class:		.: (0.000679)
    test_parse_data_packet_should_read_string_from_packet_and_return_data_in_hash:.: (0.000380)
    test_parse_handle_packet_should_read_string_from_packet_and_return_handle_in_hash:.: (0.000332)
    test_parse_name_packet_should_use_correct_name_class:		.: (0.000558)
    test_parse_status_packet_should_read_long_from_packet_and_return_code_in_hash:.: (0.000318)
    test_read_should_send_read_packet:					.: (0.000449)
    test_readdir_should_send_readdir_packet:				.: (0.000472)
    test_readlink_should_raise_not_implemented_error:			.: (0.000371)
    test_realpath_should_send_realpath_packet:				.: (0.000413)
    test_remove_should_send_remove_packet:				.: (0.000474)
    test_rename_should_raise_not_implemented_error:			.: (0.000338)
    test_rmdir_should_send_rmdir_packet:					.: (0.000407)
    test_setstat_should_translate_hash_to_attributes_and_send_setstat_packet:.: (0.000514)
    test_stat_should_ignore_flags_parameter:				.: (0.000419)
    test_stat_should_send_stat_packet:					.: (0.000439)
    test_symlink_should_raise_not_implemented_error:			.: (0.000349)
    test_unblock_should_raise_not_implemented_error:			.: (0.000435)
    test_version:							.: (0.000294)
    test_write_should_send_write_packet:					.: (0.000426)
    Protocol::V02::TestBase: 
      test_rename_should_ignore_flags_parameter:				.: (0.000739)
      test_rename_should_send_rename_packet:				.: (0.000465)
      test_version:							.: (0.000315)
      Protocol::V03::TestBase: 
        test_readlink_should_send_readlink_packet:			.: (0.000723)
        test_symlink_should_send_symlink_packet:				.: (0.000491)
        test_version:							.: (0.000333)
        Protocol::V04::TestBase: 
          test_fstat_should_send_fstat_packet:				.: (0.000852)
          test_fstat_with_custom_flags_should_send_fstat_packet_with_given_flags:.: (0.000646)
          test_lstat_should_send_lstat_packet:				.: (0.000659)
          test_lstat_with_custom_flags_should_send_lstat_packet_with_given_flags:.: (0.000633)
          test_parse_attrs_packet_should_use_correct_attributes_class:	.: (0.000752)
          test_parse_name_packet_should_use_correct_name_class:		.: (0.000591)
          test_stat_should_send_stat_packet:				.: (0.000538)
          test_stat_with_custom_flags_should_send_stat_packet_with_given_flags:.: (0.000453)
          test_version:							.: (0.000327)
          Protocol::V05::TestBase: 
            test_open_with_a_plus_should_translate_correctly:		.: (0.000949)
            test_open_with_a_should_translate_correctly:			.: (0.000544)
            test_open_with_attributes_converts_hash_to_attribute_packet:	.: (0.000563)
            test_open_with_numeric_flag_should_accept_IO_constants:	.: (0.000600)
            test_open_with_r_plus_should_translate_correctly:		.: (0.000541)
            test_open_with_r_should_translate_correctly:			.: (0.000523)
            test_open_with_rb_should_translate_correctly:		.: (0.000546)
            test_open_with_w_plus_should_translate_correctly:		.: (0.000529)
            test_open_with_w_should_translate_correctly:			.: (0.000558)
            test_rename_should_send_rename_packet:			.: (0.000520)
            test_rename_with_flags_should_send_rename_packet_with_flags:	.: (0.000668)
            test_version:						.: (0.000971)
            Protocol::V06::TestBase: 
              test_block_should_send_block_packet:			.: (0.000914)
              test_link_should_send_link_packet:				.: (0.000508)
              test_parse_attrs_packet_should_use_correct_attributes_class:.: (0.000733)
              test_symlink_should_send_link_packet_as_symlink:		.: (0.000491)
              test_unblock_should_send_unblock_packet:			.: (0.000477)
              test_version:						.: (0.000325)
              Protocol::V06::TestExtended: 
                test_hash_should_send_hash_packet:			.: (0.000797)
                test_hash_should_send_hash_packet_with_block_size:	.: (0.000553)
                test_hash_should_send_hash_packet_with_valid_block_size:	.: (0.000604)
                test_home_should_send_home_directory_packet:		.: (0.000521)
                test_md5_should_send_md5_hash_packet:			.: (0.000547)
                test_space_available_should_send_space_available_packet:	.: (0.000572)
                test_version:						.: (0.000401)

Finished in 0.051579164 seconds.
------------------------------------------------------------------------------------------------
79 tests, 150 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed
------------------------------------------------------------------------------------------------
1531.63 tests/s, 2908.15 assertions/s

[nabilbendafi]

I'v updated my PR after rebasing #59 work.

[nabilbendafi]

I misclicked on close and comment. PR reopened. Sorry.

[mfazekas]

@nabilbendafi thank for that! Looks awesome. Can you also add the usecase for it as a github comment?

[nabilbendafi]

Use-cases can be multiple:

• The most relevant one for me would be to use check-file-handle or md5-hash-handle in order to implement an rsync-like SFTP transfert, where you can sync local and remote file(s)/folder(s) while only sending missing data chunks. Saving you some bandwidth.
• Another one would be to prevent the upload(Net::SFTP::Operations) of data, if there is not enough disk space on remote side.

[mfazekas]

Do you think we can add end to end tests like we have in net-ssh?

https://github.com/net-ssh/net-ssh/tree/master/test/integration

[nabilbendafi]

Well, it's good idea. Though no integration tests are already shipped with Net::SFTP.
Would it be better to first introduce integration tests via another PR, covering already available functions (upload, download, ...) and then come back to this one ?

[mfazekas]

Ok sounds fine. All those extensions in this pr are supported by openssh? And you tried all them, against a real server?

[nabilbendafi]

How disappointed I am (and not the only one, obviously: SFTP and OpenSSH's "do as we please") to read that even if such extensions are part of latest RFCs, they are not part of OpenSSH code !

Note that OpenSSH's sftp and sftp-server implement revision 3 of the SSH
filexfer protocol described in:

http://www.openssh.com/txt/draft-ietf-secsh-filexfer-02.txt

Newer versions of the draft will not be supported, though some features
are individually implemented as extensions described below.

(source: http://web.mit.edu/freebsd/head/crypto/openssh/PROTOCOL)

Instead, they use their own vendor specific extensions, such as statvfs@openssh.com.

I guest that I'll have to test against a SFTP server that supports them, such as Apache Mina SSHD