HIP-1383: Enhanced Error Responses

Abstract

This HIP proposes to modify the response payload of all Hiero APIs to include a new Outcome message that can be used to return more details about the result of the API call. The goal of this addition is to improve developer experience, reduce debugging time, and support automation and SDK failure handling. The proposal in this HIP does not remove or modify existing structures, only adds new ones. Because of this, backwards compatibility should be maintained.

While this proposal modifies the response for all requests, it is primarily focused on error responses. The proposed changes will be made extendable, such that if in the future there is any demand for including more detailed information for non-failure responses, those changes can be incorporated without much friction.

Motivation

Today, when using the Hiero APIs, some error responses lack context and details to help users understand what failed and what corrective actions, if any, can be taken. The only piece of error information returned today is a simple enum: proto.ResponseCodeEnum. These enum values vary in their usefulness. For example, some values are self-explanatory such as INSUFFICIENT_PAYER_BALANCE and PAYER_ACCOUNT_NOT_FOUND. However, others are more vague and don’t reveal details about what failed, such as INVALID_TRANSACTION and FAIL_INVALID. Because these codes are generic and don’t contain the failure details, users are left with little insight to resolve the issue.

Users (both in the community and Hashgraph developers) often have trouble diagnosing errors that occur using the Hiero APIs because the responses do not include any details about what failed and how to resolve the issue. When a failure happens, users should not need to guess what generic error codes actually mean. This proposal allows for errors to be more descriptive.

Rationale

Results of a request can be found in multiple places:

• Transaction/Query response: When a transaction/query is submitted, a response is returned immediately - for either success or failure. The failures in this path are limited to pre-check failures such as initial validation issues.
• Consensus Node Record Cache: A short-lived cache local to the consensus node that contains the result of successfully submitted transactions. This information can be retrieved by a user when a TransactionGetReceipt query is performed.
• Mirror Node and Record Stream: Permanent storage of submitted transaction results.

In either case, for failures only the simple ResponseCodeEnum is made available.

In order to provide additional context into failures, this design proposes adding a new message called an Outcome to the response for all Hiero APIs. This new message will contain detailed error codes along with optional messages and retry information. This new message is additive to the existing response payloads and does not alter any existing fields or messages already included.

User stories

• As a user of the Hiero APIs, when a response is received I want to quickly determine if the request was successful or not.
• As a user of the Hiero APIs, if a request I made failed, I want to be able to easily diagnose the problem and determine how to resolve the failure.
• As a user of the Hiero APIs, if a request I made failed, I would like to know if I can retry the request.

Specification

There are two components to this proposal:

1. The new outcome message that is included in each response.
2. An error code lookup dictionary that maps detail codes to user-friendly messages.

The Outcome Message

To assist users in recovering from failures, a new response structure is proposed to be included in all responses. This new structure is called an “outcome.” This outcome detail will contain four new components: a high level outcome type, the original response code, and a response error detail that contains detailed information about the failure and retry information.

• Outcome type: Enum that represents a very high level classification of the response. Values include success, client error, and server error. The purpose of this field is to quickly allow the user to determine if the response is successful or a failure, and in the case of failures, whether the failure was a user error or an internal error that is likely outside the control of the user.
• Base response code: Duplicate of the existing response code that is returned today. This is duplicated solely for the purpose of keeping all outcome information together and not fragmented across the response payload.
• Error details: New message/object that contains detailed information about the failure reason and potential retry options.
  • Response detail code: This is an integer code that is similar to the base response code, but is more detailed. It can be thought of as a sub-code of the base response code. These codes are not part of defined enumeration, like the ResponseCodeEnum. This allows for adding new detail codes without the need of additional HIPs.
  • Message: A custom, parameterized message that includes more information if the included response detail code does not contain enough context. In most cases the response detail code should provide enough information for the user to resolve the failure (if it is a user/client error) and thus this message will be empty/null.
    For cases when a message is included, it is the responsibility of the implementor to ensure the message is not unnecessarily large and does not include privileged/sensitive information. Furthermore, for cases where message parameters are used, it is recommended that parameter values be limited to 25 characters. If a value exceeds this length then it shall be truncated and an ellipses (…) will be appended to the value.
    Lastly, this message is classified as debug information that may change over time. While the goal of the message would be to remain static, future changes to the message may be made. Consumers of the message should NOT code against or otherwise rely on the contents/format of the message - e.g. basing automations on the message contents.
  • Retry information: Information about whether the failing request can be retried, and if so, what sort of retry is permitted. For example, in some failure cases a retry isn’t possible - such as when the request was malformed and will always fail, and in other cases a delayed retry may allow the same request to succeed - such as when the caller is being throttled/rate limited. Note: In the context of this HIP, the term “retry” means resubmitting the request WITHOUT any modification to the request payload. If a request is modified between attempts, then those requests are new, unique requests and NOT considered retries.

The proposed new Outcome definition is as follows:

/**
 * Message that represents the outcome of a submitted request.
 *
 * If the request failed, then the outcome MAY include the
 * `response_error_detail` field that contains more information about the
 * failure.
 */
message Outcome {
    /**
     * The high-level outcome classification of the request.
     */
    OutcomeType outcome_type = 1;
    /**
     * The base response code associated with the outcome of the request.
     */
    proto.ResponseCodeEnum response_code = 2;
    /**
     * Additional details of the request failure, if it did fail.
     *
     * If the outcome is of type failure, then the outcome MAY include additional
     * error details, otherwise this field will be null.
     */
    ResponseErrorDetail response_error_detail = 3;
}

/**
 * Message that contains detailed error information.
 */
message ResponseErrorDetail {
    /**
     * The detailed error code associated with the failure. If a detail code of 0
     * is present, it means no detail was included.
     */
    uint32 detail_code = 1;
    /**
     * User-friendly message that contains additional context about the failure.
     * This field is OPTIONAL.
     */
    optional string message = 2;
    /**
     * List of message parameters associated with the included message. This
     * field is OPTIONAL.
     *
     * This field SHALL be empty if there is no message associated with the
     * failure or if the message contains no parameterized values.
     */
    repeated ResponseMessageParameter message_parameters = 3;
    /**
     * Retry information associated with the outcome.
     */
    RetryHint retry_hint = 4;
}

/**
 * Message that represents a parameterized value.
 */
message ResponseMessageParameter {
    /**
     * The index of the parameter captured by this message. Indexes start at 1.
     */
    uint32 index = 1;
    /**
     * The parameterized value that SHOULD be substituted into the accompanying
     * failure message at the specified index.
     *
     * If the parameterized value length is over 25 characters, then the value
     * MUST be truncated to a length of 25 characters and an ellipses (...) MUST
     * be appended.
     */
    string value = 2;
}

/**
 * Information about whether the request can be retried, without modification.
 */
message RetryHint {
    /**
     * The type of retry permitted based on the outcome of a request.
     */
    RetryType retry_type = 1;
    /**
     * If a delayed retry is permitted, then this field SHALL contain the duration
     * (in seconds) should should be waited before attempt to retry the request.
     */
    optional uint32 delay_seconds = 2;
}

/**
 * Enumeration of the various types of retry permitted.
 */
enum RetryType {
    /**
     * A retry hint is not specified.
     */
    UNSPECIFIED_RETRY = 0;
    /**
     * The request MAY be retried, with modification, immediately.
     */
    IMMEDIATE_RETRY = 1;
    /**
     * The request MAY be retried, but there is not enough information available
     * to determine if it will be successful.
     */
    INDETERMINATE_RETRY = 2;
    /**
     * The request SHOULD NOT be retried without modification.
     */
    NO_RETRY = 3;
    /**
     * The request MAY be retried after a delay. The outcome details SHOULD
     * include a delay duration that can be used to determine when to try again.
     */
    DELAYED_RETRY = 4;
}

/**
 * An enumeration of possible outcome types.
 */
enum OutcomeType {
    /**
     * The request outcome is unknown.
     */
    UNSPECIFIED = 0;
    /**
     * The request was successfully completed.
     */
    SUCCESS = 1;
    /**
     * The request failed due to a client-side issue.
     */
    CLIENT_ERROR = 2;
    /**
     * The request failed due to a server-side issue.
     */
    SERVER_ERROR = 3;
    /**
     * The outcome of the request is not yet known because it has not been
     * completed.
     */
    PENDING = 4;
}

The Error Code Dictionary

While the outcome message includes a detailed error code, the code itself is a simple integer. Because of this, without a method to lookup what the error code means, the code is useless. Therefore, as part of this proposal, a lookup table of all the error codes and a user-friendly associated string describing the code is included.

Within the consensus node application, detailed error codes will always be associated with a message that describes the code. These codes and messages will be collated together to form one document with all the information. This file will be formatted such that each line represents one detail code with the format: <code>:<message>. For example, the file may contain the following:

1:An error has occurred.
2:Signature map is missing.
3:Account balance does not have enough funds to complete the transaction.

It is recommended that all networks implementing this feature, should include this error code lookup information in a manner that allows their users to easily search.

The lookup information should also be included in new SDKs such that when the SDK receives a detailed error code, it can translate the code into a user-friendly message that it can then present to the user.

New detail error codes can be added between releases. When a new release is made available, a new lookup file will be generated. Once a code is assigned a failure case, it cannot be removed or altered such that the meaning of the code is different. For example, if error code 42 is defined as “Account information cannot be found”, subsequent releases may not change the message to something vastly different like “Updated token supply must not be negative.” Messages may be altered to improve clarity, but they should always represent the same failure.

Updates to Existing Responses

The new outcome message will be added to the following response messages: ResponseHeader, TransactionResponse, and TransactionReceipt.

Query Responses

When a query request is made, the query response will have the ResponseHeader message updated to include the outcome message:

// Current Definition
message ResponseHeader {
    ResponseCodeEnum nodeTransactionPrecheckCode = 1;
    ResponseType responseType = 2;
    uint64 cost = 3;
    bytes stateProof = 4;
}

// Proposed Definition
message ResponseHeader {
    ResponseCodeEnum nodeTransactionPrecheckCode = 1;
    ResponseType responseType = 2;
    uint64 cost = 3;
    bytes stateProof = 4;
    Outcome outcome = 5;
}

Transaction Responses

When a transaction is submitted, the response will have the TransactionResponse message updated to include the outcome message:

// Current Definition
message TransactionResponse {
    ResponseCodeEnum nodeTransactionPrecheckCode = 1;
    uint64 cost = 2;
}

// Proposed Definition
message TransactionResponse {
    ResponseCodeEnum nodeTransactionPrecheckCode = 1;
    uint64 cost = 2;
    Outcome outcome = 3;
}

Transaction Receipts

When a transaction’s result is queried via the get receipt API, the TransactionReceipt will include the new outcome message:

// Current Definition
message TransactionReceipt {
    ResponseCodeEnum status = 1;
    AccountID accountID = 2;
    FileID fileID = 3;
    ContractID contractID = 4;
    ExchangeRateSet exchangeRate = 5;
    TopicID topicID = 6;
    uint64 topicSequenceNumber = 7;
    bytes topicRunningHash = 8;
    uint64 topicRunningHashVersion = 9;
    TokenID tokenID = 10;
    uint64 newTotalSupply = 11;
    ScheduleID scheduleID = 12;
    TransactionID scheduledTransactionID = 13;
    repeated int64 serialNumbers = 14;
    uint64 node_id = 15;
}

// Proposed Definition
message TransactionReceipt {
    ResponseCodeEnum status = 1;
    AccountID accountID = 2;
    FileID fileID = 3;
    ContractID contractID = 4;
    ExchangeRateSet exchangeRate = 5;
    TopicID topicID = 6;
    uint64 topicSequenceNumber = 7;
    bytes topicRunningHash = 8;
    uint64 topicRunningHashVersion = 9;
    TokenID tokenID = 10;
    uint64 newTotalSupply = 11;
    ScheduleID scheduleID = 12;
    TransactionID scheduledTransactionID = 13;
    repeated int64 serialNumbers = 14;
    uint64 node_id = 15;
    Outcome outcome = 16;
}

Impact on Block and Mirror Nodes

Since the new outcome data will be included on things like the transaction receipt, they will be sent to mirror and block nodes. This will increase storage requirements. The actual number of additional storage per transaction is dependent on whether the transaction was successful or failed, and in the case of failures to what extent details about the failure are included.

All responses, regardless of success or failure, will have a minium of 4 additional bytes. The most common failure responses will incur 8 to 14 bytes of additional storage. If a failure response includes things like debug messages and message parameters, the size may exceed 50 bytes.

Some examples:

• Simple outcome with just the outcome type and response code
  Bytes: 4

  outcome: {
      outcome_type: CLIENT_ERROR,
      response_code: BUSY
  }
  

• Simple outcome with a detail code
  Bytes: 8

  outcome: {
      outcome_type: CLIENT_ERROR,
      response_code: FAIL_INVALID,
      response_code_detail: {
          detail_code: 35
      }
  }
  

• Outcome with a detail code and retry hint
  Bytes: 12

    outcome: {
        outcome_type: CLIENT_ERROR,
        response_code: FAIL_INVALID,
        response_code_detail: {
            detail_code: 35,
            retry_hint: {
                retry_type: NO_RETRY
            }
        }
    }
  

• Outcome with a detail code and a delay retry hint
  Bytes: 14

    outcome: {
        outcome_type: CLIENT_ERROR,
        response_code: FAIL_INVALID,
        response_code_detail: {
            detail_code: 35,
            retry_hint: {
                retry_type: DELAYED_RETRY,
                delay_seconds: 30
            }
        }
    }
  

• Outcome with a short message
  Bytes: 37

    outcome: {
        outcome_type: CLIENT_ERROR,
        response_code: FAIL_INVALID,
        response_error_detail: {
            detail_code: 35,
            message: "Something bad happened.",
            retry_hint: {
                retry_type: INDETERMINATE_RETRY
            }
        }
    }
  

• Outcome with a short message that includes one message parameter and simple retry
  Bytes: 48

    outcome: {
        outcome_type: CLIENT_ERROR,
        response_code: FAIL_INVALID,
        response_error_detail: {
            detail_code: 35,
            message: "Balance must be below {1}",
            message_parameters: [
                {
                    index: 1,
                    value: "100"
                }
            ],
            retry_hint: {
                retry_type: NO_RETRY
            }
        }
    }
  

• Outcome with a short message that includes one message parameter and delay retry
  Bytes: 50

    outcome: {
        outcome_type: CLIENT_ERROR,
        response_code: FAIL_INVALID,
        response_error_detail: {
            detail_code: 35,
            message: "Balance must be below {1}",
            message_parameters: [
                {
                    index: 1,
                    value: "100"
                }
            ],
            retry_hint: {
                retry_type: DELAYED_RETRY,
                delay_seconds: 30
            }
        }
    }
  

Impact on SDK

Given the inclusion of the new outcome message and associated error code lookup file, SDKs may choose to show this new information to users. No breaking changes are proposed that would negatively impact existing SDKs. Additionally, SDKs may be updated at the owner’s discretion and do not need to be updated immediately once the consensus node application begins returning the new information.

Backwards Compatibility

This proposal does not alter any existing fields on the response messages being updated. Furthermore, this design does not propose altering any existing response codes. Given that only a new field is being added, backwards compatibility should not be impacted.

Security Implications

Information Disclosure

These proposed changes will introduce more details around failure paths instead of generic error codes. This information MAY include publicly available information such as public key fragments, account IDs, and so on. It is difficult if not impossible to fully prevent more sensitive information from being included in the payload in this proposal. It will be the responsibility of the person or persons incorporating this feature into the Hiero APIs to ensure that no sensitive data is included in the outcome payload.

Denial-of-Service/Undue Load Risk

With the additional outcome data, the size of the responses will be larger and this is especially true if a message is included. This larger size may increase the risk of resource exhaustion, however sensible message size limitations should limit this risk. If a malicious user was to uncover a failure path that included a large detail message or with many message parameters, it would be possible for them to force the failure at a high rate which could cause significant resource usage since the payload is held in memory and in the record/block streams. If rate limiting is operating properly, this attack vector should be limited.

Data at Rest

The new outcome payload will be included in the record and block streams. Privileged/sensitive information is not intended to be included in the outcome data, but if such data was included then it would be persisted forever and would introduce a potential security issue. Safeguards and additional scrutiny must be used whenever an outcome includes things like a message and message parameter values.

Limitations

• This proposal does NOT propose that every failure scenario will be enhanced, at the very least not initially. Some existing failure responses already contain enough context a user may need to correctly identify the problem. As such, these cases may not be updated to provide much more detailed information. Furthermore, it is assumed that updating existing failure paths to use this proposal will be iterative - meaning that the most generic/ambiguous errors should be enhanced first and other errors will be enhanced on a needs basis.
• This proposal does NOT define all possible detail error codes, debug messages, or scenarios in which one retry may be used over another. These are considered implementation details that will be unique to each failure scenario. For failure scenarios that are not enhanced, default values such as “UNKNOWN” or “UNSPECIFIED” will be used as placeholders until they are explicitly enhanced, if done at all.
• This proposal does NOT define how Hedera-managed SDKs will use this new information. This proposal simply defines the new data that will be available for the SDKs to use.

How to Teach This

The new outcome message included in responses contains more detailed information about failures and may contain information about how they can be resolved and whether the request can be retried, without modification.

Reference Implementation

The reference implementation must be complete before any HIP is given the status of “Final.” The final implementation must include test code and documentation.

Rejected Ideas

N/A

Open Issues

N/A

References

N/A

Copyright/license

This document is licensed under the Apache License, Version 2.0 — see LICENSE or https://www.apache.org/licenses/LICENSE-2.0.

Citation

Please cite this document as: